Opened 3 years ago

Last modified 7 weeks ago

#22509 needs_review enhancement

Install packages in temporary root before copying to $SAGE_LOCAL — at Version 10

Reported by: embray Owned by:
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: build Keywords:
Cc: mkoeppe Merged in:
Authors: Erik Bray Reviewers:
Report Upstream: N/A Work issues:
Branch: u/embray/build/destdir (Commits) Commit: e2941230aea00481d6cf1f58992faa006a7b63f2
Dependencies: #23059 #23096 Stopgaps:

Description (last modified by embray)

Currently we build packages, in most cases, by setting --prefix=$SAGE_LOCAL (in ./configure, or the equivalent in non-autoconf packages), and then installing directly to that prefix.

This is fine to keep as is. But with automake packages we can also set DESTDIR (https://www.gnu.org/software/automake/manual/html_node/DESTDIR.html#DESTDIR) to an alternative (typically temporary) directory into which to install. This has a couple advantages:

  • Although it's not as usual for make install to fail after a successful make, it can happen. This prevents messy partial installs in the case of a failed make install run.
  • It gives us the opportunity to make an exact list of the files that were installed. This is a prerequisite to improving package uninstallation/reinstallation in sage-dist (see #22510).

This is also standard operating procedure in most other packaging systems, so it would be good for Sage to adopt too. It is also already possible for Python packages (e.g. pip install --root), and shouldn't be too hard to patch into whatever packages don't support it out of the box for whatever reason.

Change History (10)

comment:1 Changed 3 years ago by mkoeppe

  • Cc mkoeppe added

comment:2 Changed 3 years ago by embray

  • Description modified (diff)
  • Milestone changed from sage-7.6 to sage-wishlist

comment:3 Changed 3 years ago by embray

  • Authors set to Erik Bray
  • Branch set to u/embray/build/destdir
  • Commit set to a25a42a02a89a20058d41d95c56efc1cd0875f26
  • Dependencies set to #23096
  • Status changed from new to needs_review

Started work on this, and converted I think all standard packages to work with a staged installation (install first to a temporary directory, then copy from the temp directory into $SAGE_LOCAL).

As I wrote in the description, most packages are easy--either they use automake, or at least a Makefile that's compatible with the DESTDIR convention, or they are Python packages in which case the --root option does the same thing. A handful of packages needed hand-tweaking.

This is currently implemented in such a way that if a package doesn't yet support staged install, it will still install just fine. The only major downside is that the installed files are not recorded yet, so they wouldn't be able to take advantage of improved uninstall.

That said, I'm going to continue converting the remaining packages. My only question is whether to continue work in this ticket, or separate the general changes (i.e. to sage-spkg) from the updates to the packages themselves.


Last 10 new commits:

3fa088aUpdate pip install to use --root
30aad2fUpdate freetype and pkgconf to use DESTDIR
b1aabaaUpdated various packages with non-standard installers or that are otherwise
223608fUpdate cephes to work with staged install.
27878c3Update openblas to use staged install
ecd214cUpdate lcalc to support staged install
3d64db8Update zn_poly, which doesn't really have a proper installer, to use staged install
609393aRoll back accidental change from dc46ea2dfa97be79d7ef8ddbe9130d4c717acc3d that broke build of Python 2 on Cygwin.
03424e1Merge remote-tracking branch 'upstream/u/embray/cygwin/ticket-23059' into u/embray/build/destdir
a25a42aUpdated the python2/3 pacakges to use staged install.

comment:4 Changed 3 years ago by embray

  • Dependencies changed from #23096 to #23059 #23096

comment:5 Changed 3 years ago by embray

By way of explanation, the idea here is that there is a variable exported by sage-spkg called $SAGE_INST_TEMP (the name is open to bikeshedding), so this is expected to be passed to each spkg-install to use an alternative install root. In principle this could be defined in each spkg-install individually, but it's less trouble to standardize in one place, and in any case sage-spkg still needs to know this path to copy files from it to $SAGE_LOCAL.

Of course, if we take this approach, the developer docs will also need to be updated with some guidance on how to implement staged installation for new packages.

comment:6 Changed 3 years ago by git

  • Commit changed from a25a42a02a89a20058d41d95c56efc1cd0875f26 to 8a621af7a5f3e10d1337c07b42d04edee6a8f1b8

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

3a32187Roll back accidental change from dc46ea2dfa97be79d7ef8ddbe9130d4c717acc3d that broke build of Python 2 on Cygwin.
8a621afUpdated the python2/3 pacakges to use staged install.

comment:7 Changed 3 years ago by git

  • Commit changed from 8a621af7a5f3e10d1337c07b42d04edee6a8f1b8 to 1391089b36f43b45fa0eb7bf04e83fc1e351dfea

Branch pushed to git repo; I updated commit sha1. New commits:

1391089Remove the '-v' from the 'cp' command.

comment:8 Changed 3 years ago by git

  • Commit changed from 1391089b36f43b45fa0eb7bf04e83fc1e351dfea to e2941230aea00481d6cf1f58992faa006a7b63f2

Branch pushed to git repo; I updated commit sha1. New commits:

e294123Add staged install support for atlas

comment:9 Changed 3 years ago by embray

  • Description modified (diff)
Last edited 3 years ago by embray (previous) (diff)

comment:10 Changed 3 years ago by embray

  • Description modified (diff)
Note: See TracTickets for help on using tickets.