Opened 10 months ago

Last modified 3 weeks ago

#31396 needs_review enhancement

relocatable wheel version of package sage_conf — at Version 15

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.5
Component: relocation Keywords:
Cc: culler, gh-kliem, dimpase, jhpalmieri, slelievre Merged in:
Authors: Matthias Koeppe Reviewers:
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/relocatable_wheel_version_of_package_sage_conf (Commits, GitHub, GitLab) Commit: 851435289634de431fc8708e0c4c59a37b578b71
Dependencies: #29039, #31409 Stopgaps:

Status badges

Description (last modified by mkoeppe)

This version of sage_conf is for making a wheel that packages the precompiled non-Python bits of the Sage distribution, making SAGE_ROOT (and thus SAGE_LOCAL=$SAGE_ROOT/local) relocatable using the method proposed in #31076, using SAGE_ROOT=/var/tmp/sage-... and a symlink.

The sage script invokes sage-config to determine SAGE_ROOT and SAGE_LOCAL. In the version of sage-config supplied by this version of sage_conf, we ensure that the symlink from /var/tmp/sage-.... to the actual install location is set.

To test:

./bootstrap && (cd src/pkgs/sage_conf-relocatable/ && tox -v -v -v)

Specifically for building a wheel on macOS that uses XCode python 3.8:

./bootstrap && (cd src/pkgs/sage_conf-relocatable/ && tox -v -v -v -e python-macos-10.15-python3.8)

This builds a 750MB wheel, an order of magnitude above the standard file size limit on PyPI.

Optional steps:

  • Also build the wheelhouse (local/var/lib/sage/wheels), at least for tricky packages...

Change History (15)

comment:1 Changed 10 months ago by mkoeppe

  • Branch set to u/mkoeppe/relocatable_wheel_version_of_package_sage_conf

comment:2 Changed 10 months ago by git

  • Commit set to 9573b7c12afc5de1de743e53ff182d20490b340c

Branch pushed to git repo; I updated commit sha1. New commits:

952fb9aFix up package_data
9573b7cCloser to working

comment:3 Changed 10 months ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Cc gh-kliem added
  • Description modified (diff)

comment:4 follow-up: Changed 10 months ago by mkoeppe

  • Description modified (diff)

On macOS, some shared libraries from homebrew still leak into the build:

find src/pkgs/sage_conf-relocatable/.tox/python/lib/python3.9/site-packages/sage_root -name "*.so" -o -name "*.dylib" | xargs otool -L
libgiac.dylib:
/usr/local/opt/gettext/lib/libintl.8.dylib
R/library/cluster/libs/*.dylib:
	/usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version 11.0.0, current version 11.0.0)
	/usr/local/opt/pcre/lib/libpcre.1.dylib (compatibility version 4.0.0, current version 4.12.0)
	/usr/local/opt/xz/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.5.0)

Definitely should disable the R build (#30383).

Also check whether setting CC=gcc -sysroot.... helps getting rid of /usr/local leaking in.

Also need to add --enable-fat-binary.

setup.py build_py should probably use a symlink from /var/tmp... into ~/.sage as well so that it is easy to test relocation after removing the symlink. Also copytree from sage_root_source should be changed so it overwrites (updates) the existing contents of SAGE_ROOT.

the tox test should include the test with otool for self-containedness.

comment:5 Changed 10 months ago by git

  • Commit changed from 9573b7c12afc5de1de743e53ff182d20490b340c to f785d3cde259485597664306e191a21ad0ab9d58

Branch pushed to git repo; I updated commit sha1. New commits:

f785d3csrc/pkgs/sage_conf-relocatable/tox.ini: Add environment macos-10.15

comment:6 Changed 10 months ago by mkoeppe

Using -isysroot helps but giac configuration needs more work - #31403

comment:7 in reply to: ↑ 4 Changed 10 months ago by mkoeppe

Replying to mkoeppe:

setup.py build_py should probably use a symlink from /var/tmp... into ~/.sage as well so that it is easy to test relocation after removing the symlink.

Does not work because sage resolves symlinks when it determines SAGE_ROOT.

comment:8 Changed 10 months ago by mkoeppe

  • Dependencies changed from #29039 to #29039, #31409

comment:9 Changed 10 months ago by git

  • Commit changed from f785d3cde259485597664306e191a21ad0ab9d58 to e56a2a956a96bd70dbb443869697e89b7fb2b58a

Branch pushed to git repo; I updated commit sha1. New commits:

6da7ba2Add variable TARGETS
b2d4ab8build/pkgs/{r,rpy2}: Downgrade to optional
8732076src/sage/repl/ipython_tests.py: Mark R interface tests # optional - rpy2
dbdf084src/sage/structure/sage_object.pyx: Mark R interface test # optional - rpy2
737b21csrc/sage/stats/r.py: Mark all 2 doctests in this file as # optional - rpy2
4403924src/sage/interfaces/r.py: Mark all tests # optional - rpy2
e56a2a9Merge branch 't/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional' into t/31396/relocatable_wheel_version_of_package_sage_conf

comment:10 Changed 10 months ago by git

  • Commit changed from e56a2a956a96bd70dbb443869697e89b7fb2b58a to a2289744baf90e3dcb288f96ef52d471ab8251bb

Branch pushed to git repo; I updated commit sha1. New commits:

a228974src/pkgs/sage_conf-relocatable/setup.py: Move the SAGE_ROOT directory away after building

comment:11 Changed 10 months ago by git

  • Commit changed from a2289744baf90e3dcb288f96ef52d471ab8251bb to c260942d3085fa369117746c635b48da5a572489

Branch pushed to git repo; I updated commit sha1. New commits:

0cac965build/pkgs/ecl: Update to 21.2.1, remove patches included in new version
6a62484delete all merged upstream patches: MR 210, MR 231
62fac00Merge branch 't/31336/upgrade_ecl_to_21_2_1' into t/31396/relocatable_wheel_version_of_package_sage_conf
c260942src/pkgs/sage_conf-relocatable/setup.py: Use --enable-download-from-upstream-url

comment:12 Changed 10 months ago by git

  • Commit changed from c260942d3085fa369117746c635b48da5a572489 to 4f17a047ec84130462cb7f91f28ed284816cfa74

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

dc5e225build/pkgs/python3/spkg-configure.m4: On macOS, if the distutils test fails, try using empty ARCHFLAGS
df17ad4tox.ini: Add configuration factors for specific homebrew python3.x
914ea58build/pkgs/python3/spkg-configure.m4: Get rid of stray AC_SUBST
986739bm4/sage_check_python_for_venv.m4: Fix reason
0b3e70dbuild/pkgs/python3/spkg-configure.m4: If PYTHON_FOR_VENV is configured to build multiarch extensions, set SAGE_ARCHFLAGS to disable it
372adcb.github/workflows/tox.yml: Update xcode versions
fc8b676SAGE_CHECK_PYTHON_FOR_VENV: Rework with less nesting
5ae8044Merge branch 'u/mkoeppe/accept__usr_bin_python3_from_xcode_12_3_on_macos_10_15__catalina_' of git://trac.sagemath.org/sage into t/31396/relocatable_wheel_version_of_package_sage_conf
c5c73a9src/pkgs/sage_conf-relocatable/tox.ini: Define environments macos-python3.8, macos-python3.7, macos-python3_xcode
4f17a04src/pkgs/sage_conf-relocatable/setup.py: Use --enable-fat-binary

comment:13 Changed 10 months ago by mkoeppe

  • Description modified (diff)

comment:14 Changed 10 months ago by git

  • Commit changed from 4f17a047ec84130462cb7f91f28ed284816cfa74 to 851435289634de431fc8708e0c4c59a37b578b71

Branch pushed to git repo; I updated commit sha1. New commits:

8514352src/pkgs/sage_conf-relocatable/tox.ini: Run delocate-listdeps

comment:15 Changed 10 months ago by mkoeppe

  • Description modified (diff)
Note: See TracTickets for help on using tickets.