Opened 21 months ago

Closed 16 months ago

Last modified 7 weeks ago

#30770 closed enhancement (fixed)

cython_aliases: Use ecl-config to determine compiler/linker flags for ecl

Reported by: mkoeppe Owned by:
Priority: critical Milestone: sage-9.3
Component: build Keywords: sd111
Cc: dimpase, mjo, gh-spaghettisalat, jhpalmieri, gh-tobiasdiez, isuruf, fbissey, arojas Merged in:
Authors: Tobias Diez, Matthias Koeppe Reviewers: Matthias Koeppe, Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 11ad40a (Commits, GitHub, GitLab) Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by gh-tobiasdiez)

We use ecl-config to determine compiler/linker flags for ecl. This is a step towards using a system-installed ecl and is needed/helpful for installing sage in virtual environments (#30371).

$ ecl-config --help
Usage: /usr/local/bin/ecl-config [OPTIONS] [LIBS]
Options: 
	[--cflags]
	[--libs|--ldflags]
Libs:
	cmp

Sample output:

> ./local/bin/ecl-config --cflags --libs
-Dlinux   -I/mnt/d/Programming/sage/local/include
-Wl,--rpath,/mnt/d/Programming/sage/local/lib -L/mnt/d/Programming/sage/local/lib -lecl -Wl,-rpath-link,/mnt/d/Programming/sage/local/lib -L/mnt/d/Programming/sage/local/lib -Wl,-rpath,/mnt/d/Programming/sage/local/lib      -ldl -lm

Change History (43)

comment:1 Changed 20 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:2 Changed 20 months ago by mkoeppe

  • Cc gh-spaghettisalat added

Ideally, of course, ecl would install a pkg-config module...

comment:3 Changed 19 months ago by mkoeppe

  • Keywords sd111 added

Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111

comment:4 Changed 18 months ago by mkoeppe

  • Cc jhpalmieri added

comment:5 Changed 18 months ago by mkoeppe

  • Cc gh-tobiasdiez added

comment:6 Changed 18 months ago by gh-tobiasdiez

  • Authors set to Tobias Diez
  • Branch set to public/build/ecl-config
  • Commit set to 3049e53618902318c8247e277d28f50c1f59f545
  • Description modified (diff)
  • Status changed from new to needs_review

The following is working for me (in the context of #30371).


New commits:

3049e53Use ecl-config to determine compiler/linker flags for ecl

comment:7 follow-up: Changed 18 months ago by mkoeppe

Following the conventions of autoconf, this variable should be called ECL_CONFIG, not ECL_CONFIG_PATH

comment:8 Changed 18 months ago by git

  • Commit changed from 3049e53618902318c8247e277d28f50c1f59f545 to 1041128342253f609a9c3b8f8efe4495bc314a2e

Branch pushed to git repo; I updated commit sha1. New commits:

1041128Rename ecl config variable

comment:9 in reply to: ↑ 7 Changed 18 months ago by gh-tobiasdiez

Replying to mkoeppe:

Following the conventions of autoconf, this variable should be called ECL_CONFIG, not ECL_CONFIG_PATH

I'm not sure if one should follow the autoconf conventions here since the variable is (right now) only used in python scripts, but I've renamed it anyway.

comment:10 Changed 18 months ago by mkoeppe

capture_output is not available on Python 3.6 - see #30758

comment:11 Changed 18 months ago by gh-tobiasdiez

  • Dependencies set to #30551

comment:12 Changed 18 months ago by mkoeppe

I think it's better to make the simple change of the code as in https://git.sagemath.org/sage.git/commit?id=a569b88f02ab4dfdd8eec6e8ad0472f1f537d2c8 instead of deferring this ticket until after we drop support for python 3.6

comment:13 follow-up: Changed 18 months ago by dimpase

I have yet to see a system that ships patched ECL, allowing one to build Maxima in Sage, etc.

comment:14 Changed 18 months ago by gh-tobiasdiez

I use this ticket successfully in #30371 to build sage in a new python virtual environment using sage's ecl precisely because the system ecl is not working.

comment:15 in reply to: ↑ 13 Changed 18 months ago by mkoeppe

Replying to dimpase:

I have yet to see a system that ships patched ECL, allowing one to build Maxima in Sage, etc.

Surely distributions that ship sage must have a suitable package: conda, arch, ...?

comment:16 Changed 18 months ago by mkoeppe

  • Cc isuruf fbissey arojas added

comment:17 follow-up: Changed 18 months ago by arojas

@@ -203,6 +204,7 @@ var('MAXIMA_FAS')
 var('SAGE_NAUTY_BINS_PREFIX',        '')
 var('ARB_LIBRARY',                   'arb')
 var('CBLAS_PC_MODULES',              'cblas:openblas:blas')
+var('ECL_CONFIG')
 
 # misc
 var('SAGE_BANNER', '')

Please add a fallback here, otherwise this will break distro packaging without sage_conf

comment:18 follow-up: Changed 18 months ago by arojas

Given that sage's maxima is patched to install a fas module in ecl's tree, how is this supposed to work with a system ecl?

comment:19 Changed 18 months ago by mkoeppe

I think we can change our maxima installation to use a different installation directoery and then pass the full path using the existing variable sage.env.MAXIMA_FAS

comment:20 in reply to: ↑ 17 Changed 18 months ago by mkoeppe

Replying to arojas:

@@ -203,6 +204,7 @@ var('MAXIMA_FAS')
 var('SAGE_NAUTY_BINS_PREFIX',        '')
 var('ARB_LIBRARY',                   'arb')
 var('CBLAS_PC_MODULES',              'cblas:openblas:blas')
+var('ECL_CONFIG')
 
 # misc
 var('SAGE_BANNER', '')

Please add a fallback here, otherwise this will break distro packaging without sage_conf

I agree, this should fall back to just 'ecl-config'

comment:21 Changed 18 months ago by gh-tobiasdiez

Good catch, I've now added a fallback!

comment:22 Changed 18 months ago by git

  • Commit changed from 1041128342253f609a9c3b8f8efe4495bc314a2e to fd05a972b62b0014995887f1372e7934866fdf24

Branch pushed to git repo; I updated commit sha1. New commits:

fd05a97Add fallback

comment:23 Changed 18 months ago by dimpase

#30551 is for 9.4, why is this one still for 9.3 ? (I'd rather switch #30551 to 9.3, though)

comment:24 Changed 18 months ago by gh-tobiasdiez

I've put #30551 now to 9.3 since I agree it would be convenient to drop python 3.6 support soon. Matthias, please feel free to put both tickets onto 9.4 if you strongly feel that 3.6 still needs to be supported.

Last edited 18 months ago by gh-tobiasdiez (previous) (diff)

comment:25 Changed 18 months ago by mkoeppe

  • Status changed from needs_review to needs_work

Let's please not use a ticket like this where we know exactly how to write python 3.6 conforming code to force a decision that needs proper discussion in #30551.

comment:26 Changed 18 months ago by git

  • Commit changed from fd05a972b62b0014995887f1372e7934866fdf24 to 6370c345114d50095e84a9b2e7917dd8cae3bfe6

Branch pushed to git repo; I updated commit sha1. New commits:

6370c34src/sage/env.py: Do not use capture_output, which requires python 3.7

comment:27 Changed 18 months ago by mkoeppe

  • Authors changed from Tobias Diez to Tobias Diez, Matthias Koeppe
  • Status changed from needs_work to needs_review

comment:28 follow-up: Changed 18 months ago by git

  • Commit changed from 6370c345114d50095e84a9b2e7917dd8cae3bfe6 to 31f1bbba244263160e887613b34111bc3a20d4f4

Branch pushed to git repo; I updated commit sha1. New commits:

31f1bbbAlso use universal_newlines instead of text, for python 3.6 compatibility

comment:29 in reply to: ↑ 28 Changed 17 months ago by dimpase

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

31f1bbbAlso use universal_newlines instead of text, for python 3.6 compatibility

Why do you push Python-3.6-needed changes to a ticket that depends on #30551 ? It just makes no sense to me.

comment:30 Changed 17 months ago by mkoeppe

  • Dependencies #30551 deleted

Thanks. I forgot to remove the dependency.

comment:31 in reply to: ↑ 18 ; follow-up: Changed 17 months ago by mkoeppe

Replying to arojas:

Given that sage's maxima is patched to install a fas module in ecl's tree, how is this supposed to work with a system ecl?

Where do distributions install compiled common lisp libraries?

comment:32 in reply to: ↑ 31 ; follow-up: Changed 17 months ago by fbissey

Replying to mkoeppe:

Replying to arojas:

Given that sage's maxima is patched to install a fas module in ecl's tree, how is this supposed to work with a system ecl?

Where do distributions install compiled common lisp libraries?

Gentoo does it in /usr/lib64/ecl-20.4.24 - as can be seen it is versioned. This is the place where ecl put a lot of its own libraries. It is determined by running the following ecl lisp formula, which I believe is the same in vanilla sage.

ecl -eval "(princ (SI:GET-LIBRARY-PATHNAME))" -eval "(quit)"

It could be interesting to know other, or how to set other, paths to install .fas files for ecl.

comment:33 in reply to: ↑ 32 Changed 17 months ago by gh-spaghettisalat

Replying to fbissey:

Replying to mkoeppe:

Replying to arojas:

Given that sage's maxima is patched to install a fas module in ecl's tree, how is this supposed to work with a system ecl?

Where do distributions install compiled common lisp libraries?

Gentoo does it in /usr/lib64/ecl-20.4.24 - as can be seen it is versioned. This is the place where ecl put a lot of its own libraries. It is determined by running the following ecl lisp formula, which I believe is the same in vanilla sage.

ecl -eval "(princ (SI:GET-LIBRARY-PATHNAME))" -eval "(quit)"

It could be interesting to know other, or how to set other, paths to install .fas files for ecl.

There is no universally agreed place to put .fas files, after all you can just use (load "/wherever/whatever.fas") to load them from any place you like. But /usr/lib64/ecl-20.4.24 makes the most sense for libraries useable for multiple projects and also allows you to call (require :whatever) to use the fas in ECL. On the other hand, if the .fas file is only useable for a single project, I would rather put it in /usr/lib64/some-other-project/ and explicitely load it from there.

comment:34 Changed 17 months ago by mkoeppe

  • Reviewers set to Matthias Koeppe, ...

comment:35 follow-up: Changed 17 months ago by fbissey

This is extremely interesting, how would you load a .fas from an arbitrary location?

comment:36 in reply to: ↑ 35 Changed 17 months ago by gh-spaghettisalat

Replying to fbissey:

This is extremely interesting, how would you load a .fas from an arbitrary location?

As I said, you just call (load "/path/to/lib.fas"). This is a standard common lisp function for loading compiled code or source code into the running image, much like dlopen loads a shared library in C.

Last edited 17 months ago by gh-spaghettisalat (previous) (diff)

comment:37 Changed 17 months ago by git

  • Commit changed from 31f1bbba244263160e887613b34111bc3a20d4f4 to 11ad40ae805a29feb59f0db4816cc21cd5be9f4f

Branch pushed to git repo; I updated commit sha1. New commits:

2ac4cc8Merge branch 'develop' of git://github.com/sagemath/sage into public/build/ecl-config
11ad40aMerge branch 'public/build/ecl-config' of git://trac.sagemath.org/sage into public/build/ecl-config

comment:38 Changed 17 months ago by mkoeppe

Let's continue the discussion on Maxima FASL installation in #29617 (spkg-configure.m4 and distros/ information for ecl)

The present ticket is about the parameters for compiling the sage.libs.ecl extension. Needs review

comment:39 Changed 17 months ago by mkoeppe

  • Priority changed from major to critical

Let's please get this in

comment:40 Changed 17 months ago by dimpase

  • Reviewers changed from Matthias Koeppe, ... to Matthias Koeppe, Dima Pasechnik
  • Status changed from needs_review to positive_review

lgtm

comment:41 Changed 17 months ago by gh-tobiasdiez

Thanks!

comment:42 Changed 16 months ago by vbraun

  • Branch changed from public/build/ecl-config to 11ad40ae805a29feb59f0db4816cc21cd5be9f4f
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:43 Changed 7 weeks ago by mkoeppe

  • Commit 11ad40ae805a29feb59f0db4816cc21cd5be9f4f deleted

Follow up in #33803

Note: See TracTickets for help on using tickets.