Opened 15 months ago

Closed 5 weeks ago

#29617 closed enhancement (fixed)

Support system installations of ECL and fix the kenzo SPKG build

Reported by: gh-thierry-FreeBSD Owned by:
Priority: critical Milestone: sage-9.4
Component: build: configure Keywords: spkg-configure; ECL; Maxima
Cc: mjo, fbissey, arojas, gh-spaghettisalat, gh-tobiasdiez, isuruf, mkoeppe, mmarco, jhpalmieri, gh-jcuevas-rozo, slelievre, vbraun Merged in:
Authors: Thierry Thomas, Dima Pasechnik, Michael Orlitzky Reviewers: Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: 5cbe423 (Commits, GitHub, GitLab) Commit: 5cbe423253c38535b854a60a349b368b96f6a860
Dependencies: #31593 Stopgaps:

Status badges

Description (last modified by slelievre)

I tried to use ECL and Maxima from system packages as suggested by #27330, but without success for the moment.

The spkg-configure.m4 are trivial (see attachments), but the problems occur during build:

In [dochtml] [manifolds] this error is produced:

RuntimeError: ECL says: The function SET-LOCALE-SUBDIR is undefined

Actually this error is reproducible out of the Sage's build mechanism:

$ ecl
> (require 'maxima)
> (in-package :maxima)
MAXIMA> (set-locale-subdir)
Condition of type: UNDEFINED-FUNCTION
The function MAXIMA::SET-LOCALE-SUBDIR is undefined.

According to Maxima's code, set-locale-subdir is defined in src/init-cl.lisp and:

    ;; If a file is declared as private-file it is loaded
    ;; from the build tree (which makes an out-of-tree
    ;; build possible), but not compiled (which means
    ;; that with ECL or ABCL it won't end up in the final image)
    ;; => no out-of-tree build for ECL.

Maybe someone with Lisp-fu could patch it, if possible?

Attachments (2)

ecl_spkg-configure.m4 (198 bytes) - added by gh-thierry-FreeBSD 15 months ago.
To be copied as build/pkgs/ecl/spkg-configure.m4
maxima_spkg-configure.m4 (219 bytes) - added by gh-thierry-FreeBSD 15 months ago.
To be copied as build/pkgs/maxima/spkg-configure.m4

Download all attachments as: .zip

Change History (88)

Changed 15 months ago by gh-thierry-FreeBSD

To be copied as build/pkgs/ecl/spkg-configure.m4

Changed 15 months ago by gh-thierry-FreeBSD

To be copied as build/pkgs/maxima/spkg-configure.m4

comment:1 Changed 15 months ago by gh-thierry-FreeBSD

Note: I also tried to use only ECL from the system packages, and build maxima from Sage, but in this case Maxima fails

  • with the default ECLDIR in sage-env the cmp lisp files are not found:
ecl -norc -eval '(progn (load "../lisp-utils/defsystem.lisp") (load "../lisp-utils/make-depends.lisp") (funcall (intern "CREATE-DEPENDENCY-FILE" :mk) "binary-ecl/maxima" "ecl-depends.mk") (quit))'
;;; Loading "/usr/ports/math/sage/work/stage/usr/local/var/tmp/sage/build/maxima-5.42.2/src/src/../lisp-utils/defsystem.lisp"
An error occurred during initialization:
Filesystem error with pathname #P"SYS:CMP.NEWEST".
  • when ECLDIR is patched to the right value, that works, but it fails when it tries to install the maxima library under this directory (unless you are root!).

comment:2 Changed 15 months ago by mjo

  • Cc mjo added

I would wait for #22191 before attempting this if I were you. We have a mountain of patches for ecl-16.1.2 that won't be present in any system copy, and ecl-16.1.3 is incompatible with sage (that's why we haven't upgraded to it after so many years). The latest v20.4.24 release is from just a few days ago, though, and with any luck it will work (unpatched) with sage and we can look for v20.4.24 on the system.

comment:3 Changed 15 months ago by gh-thierry-FreeBSD

OK, in FreeBSD the ECL system package is at 16.1.3!

I did not know that it was incompatible with sage, maybe this is the source of the problem, thanks.

comment:4 Changed 15 months ago by gh-thierry-FreeBSD

  • Dependencies set to #22191

comment:5 Changed 14 months ago by gh-thierry-FreeBSD

Note: applying patches from build/pkgs/ecl/patches and build/pkgs/maxima/patches to the FreeBSD ports, I have been able to use them as system packages. For ECL 16.1.3, the patches from #22191 are required.

comment:6 Changed 12 months ago by dimpase

perhaps one can get FreeBSD's ECL bumped to v20.4.24 ? Well, we do ship a lot of patches for it in #22191 - but they are all upstream patches.

comment:7 Changed 12 months ago by dimpase

  • Authors changed from gh-thierry-FreeBSD to Thierry Thomas
  • Dependencies changed from #22191 to #28991

comment:8 Changed 11 months ago by dimpase

  • Authors changed from Thierry Thomas to Thierry Thomas, Dima Pasechnik
  • Branch set to u/dimpase/packages/eclconfig
  • Commit set to 05ae4d4666921c9ecee3dbef6a659de4b7f1adb8

New commits:

05ae4d4adding spkg-configure for ecl

comment:9 Changed 11 months ago by dimpase

It appears that perhaps Gentoo's ecl misses a number of important patches - in particular it can't build maxima.fas (some path mixup).

comment:10 Changed 11 months ago by dimpase

  • Milestone changed from sage-wishlist to sage-9.3

comment:11 Changed 11 months ago by mjo

I'm not an ecl maintainer, so we will probably have to wait for the next point release of ecl to address all of the bugs that sage fixes with patches. Afterwards the spkg-configure.m4 should check for a version newer than that patch release.

If we add this spkg-configure now, we're going to be in a position similar to the one we were in with pari a few months ago. The upstream release is unsuitable, so we need to test for all of the bugs fixed by the custom patches so that we don't accept a distro version that's missing them.

comment:12 Changed 10 months ago by git

  • Commit changed from 05ae4d4666921c9ecee3dbef6a659de4b7f1adb8 to 3b7c2f6e86986f42cc0b44c69882563929e40bad

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3b7c2f6adding spkg-configure for ecl

comment:13 Changed 10 months ago by git

  • Commit changed from 3b7c2f6e86986f42cc0b44c69882563929e40bad to 301800f3e0552b5d9e997ff1f4f831ab53d76137

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

301800fadding spkg-configure for ecl

comment:14 Changed 10 months ago by git

  • Commit changed from 301800f3e0552b5d9e997ff1f4f831ab53d76137 to 83b9f9df98604504fad0844865aefc5a8f884854

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

83b9f9dadding spkg-configure for ecl

comment:15 Changed 10 months ago by git

  • Commit changed from 83b9f9df98604504fad0844865aefc5a8f884854 to d5a4fc0c7bd6db2c511f630c4e25ffcc9e0d6a76

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

d5a4fc0adding spkg-configure for ecl

comment:16 Changed 10 months ago by git

  • Commit changed from d5a4fc0c7bd6db2c511f630c4e25ffcc9e0d6a76 to aea576829d9fda9402ca45c28e80d0f154cd96bc

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

aea5768adding spkg-configure for ecl

comment:17 Changed 10 months ago by git

  • Commit changed from aea576829d9fda9402ca45c28e80d0f154cd96bc to 341e99ce5cd2e7d7e499c5b94bdb522a9b9d3a09

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

341e99cadding spkg-configure for ecl

comment:18 Changed 6 months ago by mkoeppe

  • Dependencies changed from #28991 to #30770

comment:19 Changed 6 months ago by mkoeppe

  • Cc fbissey arojas gh-spaghettisalat gh-tobiasdiez added
  • Summary changed from spkg-configure.m4 for ecl and maxima to spkg-configure.m4 and distros/ information for ecl

I've narrowed the ticket to ECL. Dealing with a system maxima can be done on a follow-up ticket.

Following up on the discussion in #30770, when using a (sufficiently patched) system ECL, our maxima install script should not attempt to install the FASL in the (typically read only) ECL system directory but rather set the configuration variable MAXIMA_FAS to the correct path.

comment:20 Changed 6 months ago by mkoeppe

  • Dependencies changed from #30770 to #30770, #31114

comment:21 Changed 6 months ago by mkoeppe

  • Branch changed from u/dimpase/packages/eclconfig to u/mkoeppe/packages/eclconfig

comment:22 Changed 6 months ago by git

  • Commit changed from 341e99ce5cd2e7d7e499c5b94bdb522a9b9d3a09 to 271f0306274ad27a4e1bf9d4533a8f05d4651f9b

Branch pushed to git repo; I updated commit sha1. New commits:

271f030adding spkg-configure for ecl

comment:23 Changed 6 months ago by mkoeppe

  • Cc isuruf added

How can we detect a sufficiently patched ECL, as is presumably shipped by conda, arch, gentoo, ...?

comment:24 Changed 6 months ago by git

  • Commit changed from 271f0306274ad27a4e1bf9d4533a8f05d4651f9b to c6062b219fadf9f695d0196d5a8b39b53e5a28aa

Branch pushed to git repo; I updated commit sha1. New commits:

c6062b2build/pkgs/ecl/distros/arch.txt: New

comment:25 Changed 5 months ago by mkoeppe

  • Dependencies changed from #30770, #31114 to #30770, #31114, #31336

comment:26 Changed 4 months ago by mjo

Now that #31336 is in, I don't think we need to detect anything other than the ECL version. If they stick to the new versioning scheme, then the following constant (defined in /usr/include/ecl/config.h, here) can probably be checked with a simple integer comparison:

#define ECL_VERSION_NUMBER 210201

comment:27 Changed 4 months ago by mkoeppe

+1

comment:28 Changed 4 months ago by mjo

  • Authors changed from Thierry Thomas, Dima Pasechnik to Thierry Thomas, Dima Pasechnik, Michael Orlitzky
  • Branch changed from u/mkoeppe/packages/eclconfig to u/mjo/ticket/29617
  • Commit changed from c6062b219fadf9f695d0196d5a8b39b53e5a28aa to ec26807e77bc1722ba249aa46b7e5a30fe1b5ea4
  • Status changed from new to needs_review

This rejects ecl-16.1.3 and accepts ecl-21.2.1 on Gentoo. I'm still waiting on the build to finish to see if it works, though.


New commits:

c1a83e4Trac #29617: check for ecl-config instead of ecl.
ec26807Trac #29617: compile a check for ECL_VERSION_NUMBER.

comment:29 Changed 4 months ago by mkoeppe

maxima install script needs adjustment, see comment 19

comment:30 follow-up: Changed 4 months ago by fbissey

In sage-on-gentoo it works fine but the following doctest failure appeared

sage -t --long --random-seed=0 /usr/lib/python3.9/site-packages/sage/interfaces/tests.py
**********************************************************************
File "/usr/lib/python3.9/site-packages/sage/interfaces/tests.py", line 34, in sage.interfaces.tests
Failed example:
    subprocess.call("echo syntax error | ecl", **kwds) in (0, 255)
Expected:
    True
Got:
    False

Not sure if it is because of any other aspect of sage-on-gentoo/Gentoo or the lone patch in vanilla sage.

comment:31 Changed 4 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:32 in reply to: ↑ 30 Changed 4 months ago by mjo

Replying to fbissey:

In sage-on-gentoo it works fine but the following doctest failure appeared ... Not sure if it is because of any other aspect of sage-on-gentoo/Gentoo or the lone patch in vanilla sage.

That's from the missing patch. When stderr is /dev/full, ecl used to go into an infinite loop. That particular problem is fixed, but the test in question actually goes a bit further and checks what happens if both stdout and stderr are full. In short: it segfaults. I reported this upstream a few hours ago not realizing we had a test for it: https://gitlab.com/embeddable-common-lisp/ecl/-/issues/634

I'd rather not add 139 (segault) to the list of allowed return codes there... so maybe we should make the ECL test weaker for the time being? The patch that sage carries is no longer being considered upstream anywhere.

comment:33 Changed 4 months ago by git

  • Commit changed from ec26807e77bc1722ba249aa46b7e5a30fe1b5ea4 to 1dcda9d4d2dcf791d30b8b34b4237c32740b8e3a

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

7e259fcbuild/pkgs/ecl: Add more to distros/
88d1ea5adding spkg-configure for ecl
04c4412build/pkgs/ecl/distros/arch.txt: New
3824393Trac #29617: check for ecl-config instead of ecl.
30f5f15Trac #29617: compile a check for ECL_VERSION_NUMBER.
dde89cfTrac #29617: substitute ecl-config path into sage_conf.
1dcda9dTrac #29617: remove failing ECL test.

comment:34 follow-up: Changed 4 months ago by mjo

What's the correct way to substitute a $SAGE_LOCAL path into sage_conf.py.in at build time? The "substitute ecl-config path into sage_conf" commit is currently broken with the SPKG because I don't know what to put in there. It works with the system copy though.

comment:35 in reply to: ↑ 34 ; follow-up: Changed 4 months ago by mkoeppe

Replying to mjo:

What's the correct way to substitute a $SAGE_LOCAL path into sage_conf.py.in at build time?

OK to just use ecl-config without path, as we may assume that SAGE_LOCAL/bin is near the front of the PATH.

comment:36 Changed 4 months ago by mkoeppe

  • Dependencies #30770, #31114, #31336 deleted

comment:37 Changed 4 months ago by git

  • Commit changed from 1dcda9d4d2dcf791d30b8b34b4237c32740b8e3a to 3c77e315ce0566f5602b426c8c12115cf64dc21d

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

fb42c4bTrac #29617: substitute ecl-config path into sage_conf.
3c77e31Trac #29617: remove failing ECL test.

comment:38 Changed 4 months ago by mjo

Ok, that worked.

We still need the maxima fix from comment:19. And I guess we should check the other packages (kenzo, fricas) that depend on ecl as well.

Last edited 4 months ago by mjo (previous) (diff)

comment:39 Changed 4 months ago by git

  • Commit changed from 3c77e315ce0566f5602b426c8c12115cf64dc21d to b923e33cefe4e9b0ee16faefe5103ab2646c14c8

Branch pushed to git repo; I updated commit sha1. New commits:

cfcf56dTrac #29617: don't set ECLDIR in sage-env.
b923e33Trac #29617: move MAXIMA_FAS variable into sage_conf.

comment:40 in reply to: ↑ 35 ; follow-up: Changed 4 months ago by mjo

Replying to mkoeppe:

Replying to mjo:

What's the correct way to substitute a $SAGE_LOCAL path into sage_conf.py.in at build time?

OK to just use ecl-config without path, as we may assume that SAGE_LOCAL/bin is near the front of the PATH.

I ran into the same problem with MAXIMA_FAS, which should default to the SAGE_LOCAL path where the Maxima SPKG installs it.

I've worked around this in the latest commit but it's pretty ugly. I made the SPKG location the default again, undoing part of #26100. That works because SAGE_LOCAL is available in env.py.

But there are two other things you might want to set MAXIMA_FAS to: an unusual but correct path, and "no path." Setting it to the correct path is easy, but now "no path" is achieved by setting MAXIMA_FAS = <junk> in sage_conf, and counting on the fallback in maxima_lib.py.

I'm certainly open to other ideas. If we decide to keep the workaround, @arojas in particular should make sure I didn't mess anything up too badly for distro packages.

This passes tests with my system ECL and SPKG maxima on Gentoo.

comment:41 in reply to: ↑ 40 Changed 4 months ago by arojas

Replying to mjo:

I'm certainly open to other ideas. If we decide to keep the workaround, @arojas in particular should make sure I didn't mess anything up too badly for distro packages.

Hi,

Looks good as far as the Arch distro package is concerned. ecl_eval("(require 'maxima \"{}\")".format(MAXIMA_FAS)) fails because the ecl modules are in the default upstream path /usr/lib/ecl-${version}, but then ecl_eval("(require 'maxima)") succeeds.

comment:42 Changed 4 months ago by dimpase

this works on my Gentoo box, with system's ECL, too.

comment:43 Changed 4 months ago by dimpase

  • Cc mkoeppe added
  • Status changed from needs_work to needs_info

Matthias, as a Lisp guru, what's your opinion on this? Maybe it all may be fixed on Lisp level?

comment:44 Changed 4 months ago by mjo

  • Work issues set to fix kenzo SPKG with system ECL

comment:45 follow-up: Changed 4 months ago by mkoeppe

I would think (hope) that the current solution is more complicated than needed.

Distributions are able to install maxima in a way that a simple (REQUIRE :MAXIMA) works in ecl. (Note that this is not just a search for a maxima.fasl but rather goes through EXT:*MODULE-PROVIDER-FUNCTIONS* into which system definition facilities such as MK:DEFSYSTEM or ASDF hook.)

If the Sage distribution has to install maxima, then we install the FASL in a custom directory and set the variable MAXIMA_FAS to the correct value in sage_conf. No defaults should be set in env.py.

Instead of the try/except in maxima_lib.py, just dispatch on empty/nonempty value of MAXIMA_FAS.

comment:46 in reply to: ↑ 45 Changed 4 months ago by mjo

Replying to mkoeppe:

If the Sage distribution has to install maxima, then we install the FASL in a custom directory and set the variable MAXIMA_FAS to the correct value in sage_conf. No defaults should be set in env.py.

Instead of the try/except in maxima_lib.py, just dispatch on empty/nonempty value of MAXIMA_FAS.

This was the first thing I tried, but I ran into the same problem as in comment:34. With no (?) way to substitute $SAGE_LOCAL at ./configure-time, I made the $SAGE_LOCAL location the default at runtime instead. That way you can effectively point MAXIMA_FAS at SAGE_LOCAL by leaving it undefined.

Pretty much all of the extra complexity comes from that. If there's a way to get $SAGE_LOCAL into MAXIMA_FAS at ./configure-time, I can make this branch a lot simpler.

comment:47 Changed 4 months ago by mkoeppe

OK, thanks for the explanation, I had forgotten about this issue.

I'll work on a general solution for this problem.

comment:48 Changed 4 months ago by mkoeppe

... see #31593.

comment:49 Changed 4 months ago by git

  • Commit changed from b923e33cefe4e9b0ee16faefe5103ab2646c14c8 to 9f1afba16f69b7e6ded12e064786fb7e4c9c155a

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

dfad5c9adding spkg-configure for ecl
d2561f3build/pkgs/ecl/distros/arch.txt: New
94e6941Trac #29617: check for ecl-config instead of ecl.
d69775aTrac #29617: compile a check for ECL_VERSION_NUMBER.
fddcc0cTrac #29617: substitute ecl-config path into sage_conf.
e562adcTrac #29617: remove failing ECL test.
fb1b8caTrac #29617: don't set ECLDIR in sage-env.
835cfc0Trac #29617: move MAXIMA_FAS to sage_conf.
7e3a721Trac #29617: modify Kenzo interface to support system ECL.
9f1afbaTrac #29617: fix kenzo SPKG build.

comment:50 Changed 4 months ago by mjo

  • Status changed from needs_info to needs_review
  • Work issues fix kenzo SPKG with system ECL deleted

Looks a lot better now, just needs testing.

comment:51 Changed 4 months ago by mkoeppe

  • Dependencies set to #31593

comment:52 Changed 4 months ago by mkoeppe

I would suggest to change the lines

            if KENZO_FAS is not None:
            if MAXIMA_FAS is not None:

to if not KENZO_FAS etc. so that it makes no difference whether these variables are undefined in sage_conf.py or defined as the empty string.

comment:53 Changed 4 months ago by git

  • Commit changed from 9f1afba16f69b7e6ded12e064786fb7e4c9c155a to 4aeb5358bfcd657d3ba91c8ae45d53300fbe820f

Branch pushed to git repo; I updated commit sha1. New commits:

4aeb535Trac #29617: treat empty MAXIMA_FAS and KENZO_FAS as None.

comment:54 Changed 4 months ago by mkoeppe

+    # assumes that $SAGE_LOCAL/bin is somewhere near
+    # the beginning of the user's PATH
+    AC_SUBST(SAGE_ECL_CONFIG, ['${prefix}'/bin/ecl-config])

this comment is no longer true and should be removed

comment:55 Changed 4 months ago by mkoeppe

+    from sage.env import KENZO_FAS
+    if not KENZO_FAS:
+        ecl_eval("(require :kenzo \"{}\")".format(KENZO_FAS))
+    else:
+        ecl_eval("(require :kenzo)")

I think this one is backwards

comment:56 Changed 4 months ago by mkoeppe

ecl_eval("(setf *load-verbose* NIL)")
-if MAXIMA_FAS is not None:
+if not MAXIMA_FAS:
     ecl_eval("(require 'maxima \"{}\")".format(MAXIMA_FAS))

this one too

comment:57 Changed 4 months ago by mkoeppe

Could you add some comments regarding the "fasb" business for Kenzo?

comment:58 Changed 4 months ago by git

  • Commit changed from 4aeb5358bfcd657d3ba91c8ae45d53300fbe820f to 161569159089bde2516bebd5f99b3b25c562ad6b

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

677cd22Trac #29617: substitute ecl-config path into sage_conf.
fe77e5eTrac #29617: remove failing ECL test.
db002e7Trac #29617: don't set ECLDIR in sage-env.
cd04a8cTrac #29617: move MAXIMA_FAS to sage_conf.
482ad99Trac #29617: modify Kenzo interface to support system ECL.
683abb9Trac #29617: fix kenzo SPKG build.
1615691Trac #29617: treat empty MAXIMA_FAS and KENZO_FAS as None.

comment:59 Changed 4 months ago by mjo

Sure, all done.

comment:60 follow-up: Changed 4 months ago by mkoeppe

So this fixes a regression for the kenzo package introduced by the ECL update?

It would be good to update the ticket description.

comment:61 Changed 4 months ago by mkoeppe

Also, run this ticket on GH Actions please so that we can see if the system packages are accepted correctly

comment:62 in reply to: ↑ 60 ; follow-up: Changed 4 months ago by mjo

  • Summary changed from spkg-configure.m4 and distros/ information for ecl to Support system installations of ECL and fix the kenzo SPKG build

Replying to mkoeppe:

So this fixes a regression for the kenzo package introduced by the ECL update?

Kenzo is optional so I wouldn't guess as to when it stopped working. Some of its tests fail; for example, sage/interfaces/kenzo.py has

EXAMPLES::                                                                                                                                                               
                                                                                                                                                                             
    sage: from sage.interfaces.kenzo import MooreSpace   # optional - kenzo                                                                                              
    sage: m24 = MooreSpace(2,4)                          # optional - kenzo                                                                                              
    sage: m24                                            # optional - kenzo                                                                                              
    [K10 Simplicial-Set]                                                    

but when I run it I get

sage: from sage.interfaces.kenzo import MooreSpace                                                                                                                           
sage: m24 = MooreSpace(2,4)                                                                                                                                                  
sage: m24                                                                                                                                                                    
[K1 Simplicial-Set]

comment:64 Changed 4 months ago by mkoeppe

  • Cc mmarco jhpalmieri gh-jcuevas-rozo added

comment:65 in reply to: ↑ 62 ; follow-up: Changed 4 months ago by mkoeppe

Replying to mjo:

Replying to mkoeppe:

So this fixes a regression for the kenzo package introduced by the ECL update?

Kenzo is optional so I wouldn't guess as to when it stopped working.

We do run CI tests on GH Actions for all optional packages (just the installation + spkg-check). See for example https://github.com/sagemath/sage/runs/2223044706 for 9.3.rc1. I do not see any build failures for kenzo. Were just some tests of the sagelib testsuite failing when kenzo is installed (the GH Actions do not test this), or what is the issue?

comment:66 in reply to: ↑ 65 ; follow-up: Changed 4 months ago by mjo

Replying to mkoeppe:

We do run CI tests on GH Actions for all optional packages (just the installation + spkg-check). See for example https://github.com/sagemath/sage/runs/2223044706 for 9.3.rc1. I do not see any build failures for kenzo. Were just some tests of the sagelib testsuite failing when kenzo is installed (the GH Actions do not test this), or what is the issue?

Digging deeper, I probably just broke kenzo with the ECL spkg.

Our ECL spkg bundles ASDF, and the bundled version works OK. But if the system copy of ECL is used and if that system copy does not bundle ASDF, then there's a backwards-incompatible API change in the ASDF make-build function which has been deprecated for a while (and now exists only for the sake of backwards-compatibility with ECL).

Ultimately the make-build function takes a type argument (was :fasl but I changed it to :fasb) and passes it to another function called select-bundle-operation. In the bundled copy of ASDF, it looks like

  ;; Find the operation that produces a given bundle-type                                                                                                                    
  (defun select-bundle-operation (type &optional monolithic)
    (ecase type
      ((:dll :shared-library)
       (if monolithic 'monolithic-dll-op 'dll-op))
      ((:lib :static-library)
       (if monolithic 'monolithic-lib-op 'lib-op))
      ((:fasl)
       (if monolithic 'monolithic-compile-bundle-op 'compile-bundle-op))
      ((:image)
       'image-op)
      ((:program)
       'program-op)))

but in my unbundled ASDF, it's

  ;; Find the operation that produces a given bundle-type                                                                                                                    
  (defun select-bundle-operation (type &optional monolithic)
    (ecase type
      ((:dll :shared-library)
       (if monolithic 'monolithic-dll-op 'dll-op))
      ((:lib :static-library)
       (if monolithic 'monolithic-lib-op 'lib-op))
      ((:fasb)
       (if monolithic 'monolithic-compile-bundle-op 'compile-bundle-op))
      ((:image)
       'image-op)
      ((:program)
       'program-op))))

To avoid having to support both, one idea that comes to mind is that we could take a newer version of asdf.lisp and copy it over the bundled version in the ECL spkg. Then :fasb would work in both of the situations above. However it would still fail if the system ECL is using a bundled asdf.lisp. Patching kenzo's compile.lisp to try both is likely the only reliable way.

comment:67 follow-up: Changed 4 months ago by mkoeppe

Does Kenzo have an active upstream maintainer?

comment:68 in reply to: ↑ 66 Changed 4 months ago by mkoeppe

Replying to mjo:

we could take a newer version of asdf.lisp and copy it over the bundled version in the ECL spkg. Then :fasb would work in both of the situations above.

As we currently do not have plans to ship additional Common Lisp libraries in the Sage distribution, I think we should not do unbundling work on ECL but stick with upstream.

Patching kenzo's compile.lisp to try both is likely the only reliable way.

Yes, I agree, this seems to be the way to go. You can use HANDLER-CASE for this http://clhs.lisp.se/Body/m_hand_1.htm

comment:69 Changed 4 months ago by mkoeppe

It would probably also be good to add a kenzo/spkg-check.in that at least tries to load it into ECL.

comment:70 in reply to: ↑ 67 ; follow-ups: Changed 4 months ago by mjo

Replying to mkoeppe:

Does Kenzo have an active upstream maintainer?

Paging @mmcarco :o

The ASDF docs say that program-op and program-system should replace make-build. Those functions are available in the bundled copy of ASDF as well, so another good long-term option would be for kenzo upstream to switch to them (if possible).

comment:71 in reply to: ↑ 70 Changed 4 months ago by mmarco

The ASDF docs say that program-op and program-system should replace make-build. Those functions are available in the bundled copy of ASDF as well, so another good long-term option would be for kenzo upstream to switch to them (if possible).

To clarify this:

What we maintain (by "we" I mean Ana Romeno, Julian Cuevas, Jose Divason and myself) is a small fork of Kenzo, modified to be compatible with ECL, easier to install, and some extra features to allow a better interface with Sage.

I am not fluent in lisp myself, but will talk about this issues with the rest of the team. Maybe we can modify the package to make it more installation friendly.

comment:72 in reply to: ↑ 70 ; follow-up: Changed 4 months ago by mkoeppe

Replying to mjo:

The ASDF docs say that program-op and program-system should replace make-build. Those functions are available in the bundled copy of ASDF as well, so another good long-term option would be for kenzo upstream to switch to them (if possible).

Yes, this sounds the like the right solution.

comment:73 in reply to: ↑ 72 ; follow-up: Changed 4 months ago by mmarco

Replying to mkoeppe:

Replying to mjo:

The ASDF docs say that program-op and program-system should replace make-build. Those functions are available in the bundled copy of ASDF as well, so another good long-term option would be for kenzo upstream to switch to them (if possible).

Yes, this sounds the like the right solution.

I have done some tests on that. Might release a new version with this change in a few days, after some testing.

So, to make sure: can you confirm that changing from

(asdf:make-build :kenzo :type :fasl :monolithic t :move-here #P".")

to

(asdf:operate 'asdf:monolithic-compile-bundle-op :kenzo)

in the compile.lisp file would solve the problem?

comment:74 Changed 4 months ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review.

comment:75 Changed 4 months ago by dimpase

  • Milestone changed from sage-9.4 to sage-9.3

is kenzo build fixed? I'm getting (on macOS/homebrew), with ecl from Homebrew

[kenzo-1.1.9] Package 'kenzo' is currently not installed
[kenzo-1.1.9] No legacy uninstaller found for 'kenzo'; nothing to do
[kenzo-1.1.9] sed: 1: "compile.lisp": command c expects \ followed by text
[kenzo-1.1.9] ********************************************************************************
[kenzo-1.1.9] failed to update make-build invocation

comment:76 Changed 4 months ago by dimpase

  • Milestone changed from sage-9.3 to sage-9.4

comment:77 in reply to: ↑ 73 Changed 4 months ago by mjo

Replying to mmarco:

I have done some tests on that. Might release a new version with this change in a few days, after some testing.

So, to make sure: can you confirm that changing from

(asdf:make-build :kenzo :type :fasl :monolithic t :move-here #P".")

to

(asdf:operate 'asdf:monolithic-compile-bundle-op :kenzo)

in the compile.lisp file would solve the problem?

It fixes the build for my system ECL (with ASDF unbundled). If it also works with sage's ECL spkg, I think we have a solution. Thanks!

comment:78 Changed 4 months ago by mmarco

I just made a new release. You can get the tarball at https://github.com/miguelmarco/kenzo/releases/download/1.1.10/kenzo-1.1.10.tar.gz

Please check that it solves the problem, and then we can update the kenzo Sage package.

comment:79 Changed 4 months ago by git

  • Commit changed from 161569159089bde2516bebd5f99b3b25c562ad6b to 2d40f36753ed0f4a93c44952730d67e06f19eb38

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

27727acbuild/pkgs/ecl/distros/arch.txt: New
38cc46dTrac #29617: check for ecl-config instead of ecl.
18cbceeTrac #29617: compile a check for ECL_VERSION_NUMBER.
444d828Trac #29617: substitute ecl-config path into sage_conf.
4061f54Trac #29617: remove failing ECL test.
edd246fTrac #29617: don't set ECLDIR in sage-env.
133aab7Trac #29617: move MAXIMA_FAS to sage_conf.
9b595c0Trac #29617: modify Kenzo interface to support system ECL.
b4ae35eTrac #29617: treat empty MAXIMA_FAS and KENZO_FAS as None.
2d40f36Trac #29617: upgrade kenzo to v1.1.10.

comment:80 Changed 4 months ago by mjo

The kenzo upgrade is now included, and my previous broken spkg-install hack removed. Everything still works with system ECL.

Github has helpfully disabled actions on my account after the last run, and I'm waiting to hear back from support. Can someone else kick off a CI run please?

comment:81 Changed 4 months ago by mkoeppe

  • Reviewers set to https://github.com/mkoeppe/sage/actions/runs/732049335

Running at https://github.com/mkoeppe/sage/actions/runs/732049335 (with a bunch of other tickets)

comment:82 Changed 4 months ago by mkoeppe

  • Reviewers changed from https://github.com/mkoeppe/sage/actions/runs/732049335 to Matthias Koeppe
  • Status changed from needs_review to positive_review

One of the bunch of tickets was broken; the failures that you see in the run are from that.

A new run is at https://github.com/mkoeppe/sage/actions/runs/733869032:

Correctly accepts system ecl on homebrew-macos-11.0-xcode_default-standard (https://github.com/mkoeppe/sage/runs/2308094353) and has clean doctests

Correctly accepts system ecl on gentoo-standard (https://github.com/mkoeppe/sage/runs/2308093679). The build completes. The numerous doctest failures are from the unrelated unfortunate situation described in #31578

Correctly accepts system ecl on fedora-34-standard (https://github.com/mkoeppe/sage/runs/2308093524), but the build fails for unrelated reasons

Correctly rejects too old system ecl on other platforms.

Build of kenzo was tested in https://github.com/mkoeppe/sage/actions/runs/732049334, looks OK.

comment:83 Changed 2 months ago by mkoeppe

  • Branch changed from u/mjo/ticket/29617 to u/mkoeppe/ticket/29617

comment:84 Changed 2 months ago by slelievre

  • Cc slelievre added
  • Commit changed from 2d40f36753ed0f4a93c44952730d67e06f19eb38 to 5cbe423253c38535b854a60a349b368b96f6a860
  • Description modified (diff)

New commits:

5161d65build/pkgs/sage_conf/src/sage_conf.py.in: Move SAGE_ROOT, SAGE_LOCAL to beginning of file; only use substitution @prefix@ once
986ca18build/pkgs/sage_conf/src/sage_conf.py.in: replace subst by replace
2078fa6Merge tag '9.4.beta0' into t/31593/configure__paths_within__sage_local___prefix__for_sage_conf
5cbe423Merge #31593

comment:85 Changed 7 weeks ago by dimpase

  • Cc vbraun added
  • Priority changed from major to critical

comment:86 Changed 5 weeks ago by vbraun

  • Branch changed from u/mkoeppe/ticket/29617 to 5cbe423253c38535b854a60a349b368b96f6a860
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.