Opened 3 years ago

Last modified 3 years ago

#21480 closed enhancement

Keep src/ clean by using --build-base when building sagelib — at Version 21

Reported by: mkoeppe Owned by:
Priority: blocker Milestone: sage-7.4
Component: build Keywords:
Cc: felixs, jdemeyer, fbissey, embray, leif, vbraun, dimpase, jhpalmieri, vdelecroix, saraedum, slabbe, nthiery, mmezzarobba Merged in:
Authors: Matthias Koeppe Reviewers:
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/keep_src__clean_by_using___build_base_when_building_sagelib (Commits) Commit: a73fa065f5030c3b260c04a7e36867fd7f89362f
Dependencies: Stopgaps:

Description (last modified by mkoeppe)

Currently, building sagelib creates the src/build directory, with subdirectories cython_debug, cythonized, lib.UNAME, temp.UNAME.

In preparation for VPATH builds of sage-the-distribution (#21469), let's keep src/ clean by using setup.py --build-base=$SAGE_BUILD_DIR/sagelib

($SAGE_BUILD_DIR defaults to $SAGE_ROOT/var/tmp/sage/build/)

Some possibly useful information:

  • Documentation on distutils (https://docs.python.org/2/install/), describing use of --build-base to do VPATH builds.
  • pip install keeps the source directory clean, building instead in a temporary directory, by copying the sources. pip install also offers options --build to select a build directory, but there are some pip issues: 2060, 2053, 804 that affect this
  • #14807 has some tricks to making VPATH builds work without copying all python source files. But it uses automake instead of setup.sh; we will not do this in our ticket.

Change History (21)

comment:1 Changed 3 years ago by fbissey

Am interesting issue is that technically cython_debug has to be installed somewhere (preferably somewhere standard) to be accessible at runtime separately from the source.

I do something in sage-on-gentoo but that's not really satisfactory.

Last edited 3 years ago by fbissey (previous) (diff)

comment:2 in reply to: ↑ description ; follow-up: Changed 3 years ago by leif

Replying to mkoeppe:

In preparation for VPATH builds of sage-the-distribution (#21469), let's keep src/ clean by using setup.py --build-base=$SAGE_ROOT/var/tmp/sage/build/sagelib

Please use --build-base=$SAGE_BUILD_DIR/sagelib.

comment:3 Changed 3 years ago by leif

... and sagelib perhaps with a version suffix.

comment:4 in reply to: ↑ 2 ; follow-up: Changed 3 years ago by mkoeppe

Replying to leif:

Replying to mkoeppe:

In preparation for VPATH builds of sage-the-distribution (#21469), let's keep src/ clean by using setup.py --build-base=$SAGE_ROOT/var/tmp/sage/build/sagelib

Please use --build-base=$SAGE_BUILD_DIR/sagelib.

Yes, the plan *after* #21469 is to use the Sage builddir -- not just for sagelib, but also for other packages.

*Before* #21469 is merged, I want to use $SAGE_ROOT/var/tmp/sage/build/sagelib to match what other packages do.

comment:5 Changed 3 years ago by mkoeppe

  • Description modified (diff)

Ah, I see what you meant, now I've found $SAGE_BUILD_DIR. Changed description accordingly.

comment:6 in reply to: ↑ 4 ; follow-up: Changed 3 years ago by leif

Replying to mkoeppe:

Replying to leif:

Replying to mkoeppe:

In preparation for VPATH builds of sage-the-distribution (#21469), let's keep src/ clean by using setup.py --build-base=$SAGE_ROOT/var/tmp/sage/build/sagelib

Please use --build-base=$SAGE_BUILD_DIR/sagelib.

Yes, the plan *after* #21469 is to use the Sage builddir -- not just for sagelib, but also for other packages.

??? SAGE_BUILD_DIR exists since years already... (Its default is $SAGE_ROOT/var/tmp/sage/build/.)


*Before* #21469 is merged, I want to use $SAGE_ROOT/var/tmp/sage/build/sagelib to match what other packages do.

comment:7 in reply to: ↑ 6 Changed 3 years ago by mkoeppe

Replying to leif:

??? SAGE_BUILD_DIR exists since years already... (Its default is $SAGE_ROOT/var/tmp/sage/build/.)

Yes, thanks, see my other comment.

comment:8 Changed 3 years ago by leif

Ah ok, race condition.

comment:9 Changed 3 years ago by mkoeppe

By the way, help with implementing this change would be appreciated. I haven't looked at the sagelib build system at all so far.

comment:10 Changed 3 years ago by mkoeppe

  • Branch set to u/mkoeppe/keep_src__clean_by_using___build_base_when_building_sagelib

comment:11 Changed 3 years ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Commit set to e399bf41d805da7ea602daa5b554e0c7ecf2e7b5

New commits:

e399bf4First, wishful step

comment:12 Changed 3 years ago by mkoeppe

  • Description modified (diff)

comment:13 Changed 3 years ago by jdemeyer

Really sagelib-$SAGE_VERSION? Do you want to fill up everybody's hard disks with Sage build directories? Not to mention that this would require to rebuild everything whenever the Sage version changes.

Last edited 3 years ago by jdemeyer (previous) (diff)

comment:14 Changed 3 years ago by jdemeyer

  • Description modified (diff)

comment:15 Changed 3 years ago by mkoeppe

Fine with me without $SAGE_VERSION; I was just following leif's suggestion in comment 3.

comment:16 follow-up: Changed 3 years ago by felixs

i understand that SAGE_BUILD_DIR is the srcdir for toplevel configure.

as long as sagelib is rooted in $(toplevel)/src, it might be less confusing to choose $SAGE_BUILD_DIR/src (not $SAGE_BUILD_DIR/sagelib) as the builddir for sagelib.

(future: replace src by sagelib, but on both ends)

i dont know exactly how the approach using setup.py will emulate VPATH builds. i think it should imitate "what automake would do", where applicable.

comment:17 in reply to: ↑ 16 Changed 3 years ago by mkoeppe

Replying to felixs:

i understand that SAGE_BUILD_DIR is the srcdir for toplevel configure.

No, it's $SAGE_ROOT/var/tmp/sage/build/

comment:18 Changed 3 years ago by felixs

ok, nevermind (i meant to write "SAGE_BUILD_DIR is the *builddir* for toplevel"). that does not seem to be the case either.

still i am wondering why "src" does not (simply) translate to "src". (sure, i am slightly autotools biased).

comment:19 Changed 3 years ago by git

  • Commit changed from e399bf41d805da7ea602daa5b554e0c7ecf2e7b5 to a73fa065f5030c3b260c04a7e36867fd7f89362f

Branch pushed to git repo; I updated commit sha1. New commits:

f3f275ePut .cython_version next to cythonized files
94a58f1Make SAGE_BUILD_DIR variable available in sage-env, not just in sage-spkg
a73fa06Use SAGE_BUILD_DIR

comment:20 Changed 3 years ago by mkoeppe

  • Status changed from new to needs_review

Here's a first version for review. It seems to work for me.

There are still some to-do items (see comments in src/Makefile):

  • I am now using --build-base, but setup.sh also depends on SAGE_CYTHONIZED, defined in src/sage/env.py.

I think it would be better if setup.sh instead inferred that location from the build-base that was passed to it.

However, setup.sh already does a lot of stuff depending on SAGE_CYTHONIZED before distutils.core.setup is even called. Can this be fixed?

I think I could use some help from the Python experts in the cc list of this ticket on this.

  • sage/libs/pari/auto_gen.pxi and sage/ext/interpreters/__init__.py still need to be taken care of in preparation for the VPATH build.

comment:21 Changed 3 years ago by mkoeppe

  • Description modified (diff)
Note: See TracTickets for help on using tickets.