Opened 7 months ago
Last modified 6 weeks ago
#31396 needs_review enhancement
relocatable wheel version of package sage_conf
Reported by:  mkoeppe  Owned by:  

Priority:  major  Milestone:  sage9.5 
Component:  relocation  Keywords:  
Cc:  culler, ghkliem, dimpase, jhpalmieri, slelievre  Merged in:  
Authors:  Matthias Koeppe  Reviewers:  
Report Upstream:  N/A  Work issues:  
Branch:  u/mkoeppe/relocatable_wheel_version_of_package_sage_conf (Commits, GitHub, GitLab)  Commit:  e4b1a959b399e81202f106740f6ce314fc16290e 
Dependencies:  #29039  Stopgaps: 
Description (last modified by )
This version of sage_conf
is for making a wheel that packages the precompiled nonPython bits of the Sage distribution, making SAGE_ROOT
(and thus SAGE_LOCAL=$SAGE_ROOT/local
) relocatable using Marc Culler's symbolic link surgery (the method proposed in #31076, using SAGE_ROOT=/var/tmp/sage...
and a symlink).
The sage
script invokes sageconfig
to determine SAGE_ROOT
and SAGE_LOCAL
. In the version of sageconfig
supplied by this version of sage_conf
, we ensure that the symlink from /var/tmp/sage....
to the actual install location is set.
So far, a wheel has been built for XCode python 3.8 on macOS 10.15. The wheel is 670MB in size, an order of magnitude above the standard file size limit on PyPI; but a file size limit increase (requested in https://github.com/pypa/pypisupport/issues/985) has kindly been granted by the PyPI team.
The wheel declares dependencies (installrequires
) to all Python packages in the Sage distribution that have extension modules. The dependencies are specific using @
to URLs on
https://github.com/sagemath/sagewheels/releases/download/9.3.rc1, where I have uploaded the binary wheels. Unfortunately, by PyPI policy, such @
references are not allowed for packages on PyPI.
Hence, the pip invocation needs to use a URL.
(See also https://twitter.com/mkoeppe_math/status/1378860285537054723)
Instructions for testing:
$ export PATH=$HOME/Library/Python/3.8/bin:/usr/bin:/bin:/usr/sbin:/sbin $ /usr/bin/python3 m pip install U user https://github.com/sagemath/sagewheels/releases/download/9.3.rc1/sagemath_standard9.3rc1cp38cp38macosx_10_15_x86_64.whl $ sage
Procedure for building the sage_conf wheel and compatible wheels:
For building wheels on macOS Catalina that uses XCode python 3.8:
$ SKIP_CONFIGURE=1 tox e localmacos10.15nohomebrewpython3_xcode  bash (tox e ...) $ (cd src/pkgs/sage_confrelocatable/ && python3 setup.py bdist_wheel platname macosx_10_15_x86_64)
or
tox e localmacos10.15nohomebrewpython3_xcode  SAGE_CHECK=no sagewheels
This creates:
 src/pkgs/sage_confrelocatable/dist/sage_conf9.3rc1cp38cp38macosx_10_15_x86_64.whl
 src/pkgs/sage_confrelocatable/sage_root/venvcpython38darwinmacosx_10_15_x86_64/var/lib/sage/wheels/*.whl
Build for macOS Big Sur:
tox e localmacos11.1nohomebrewpython3_xcode  SAGE_CHECK=no sagewheels
Build for Linux:
$ TARGETS_PRE=Makefile tox e dockermanylinux2_24minimalpython3.9  sagewheels
Followup steps:
 build wheels for
manylinux
usingtox e dockermanylinux....
 wheel is much too big: 2.3GB
 auditwheel complains:
RuntimeError: Invalid binary wheel, found the following shared library/libraries in purelib folder:
; so we should replace our current abuse of package data (install_data
) by platlib (install_lib
)
 disable building the dependencies of the sphinx packages; remove unneeded static libraries; remove duplicated/triplicated shared libraries if possible;
 #31602: split out
sage_root
fromsage_conf
and use installrequires in thesage_conf
sdist/wheel with @ links to GH releases to thesage_root
wheel and the built nonany
wheels.
Change History (120)
comment:1 Changed 7 months ago by
 Branch set to u/mkoeppe/relocatable_wheel_version_of_package_sage_conf
comment:2 Changed 7 months ago by
 Commit set to 9573b7c12afc5de1de743e53ff182d20490b340c
comment:3 Changed 7 months ago by
 Cc ghkliem added
 Description modified (diff)
comment:4 followup: ↓ 7 Changed 7 months ago by
 Description modified (diff)
On macOS, some shared libraries from homebrew still leak into the build:
find src/pkgs/sage_confrelocatable/.tox/python/lib/python3.9/sitepackages/sage_root name "*.so" o name "*.dylib"  xargs otool L libgiac.dylib: /usr/local/opt/gettext/lib/libintl.8.dylib R/library/cluster/libs/*.dylib: /usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version 11.0.0, current version 11.0.0) /usr/local/opt/pcre/lib/libpcre.1.dylib (compatibility version 4.0.0, current version 4.12.0) /usr/local/opt/xz/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.5.0)
Definitely should disable the R build (#30383).
Also check whether setting CC=gcc sysroot....
helps getting rid of /usr/local
leaking in.
Also need to add enablefatbinary
.
setup.py build_py
should probably use a symlink from /var/tmp...
into ~/.sage
as well so that it is easy to test relocation after removing the symlink.
Also copytree
from sage_root_source
should be changed so it overwrites (updates) the existing contents of SAGE_ROOT.
the tox test should include the test with otool
for selfcontainedness.
comment:5 Changed 7 months ago by
 Commit changed from 9573b7c12afc5de1de743e53ff182d20490b340c to f785d3cde259485597664306e191a21ad0ab9d58
Branch pushed to git repo; I updated commit sha1. New commits:
f785d3c  src/pkgs/sage_confrelocatable/tox.ini: Add environment macos10.15

comment:6 Changed 7 months ago by
Using isysroot helps but giac configuration needs more work  #31403
comment:7 in reply to: ↑ 4 Changed 7 months ago by
Replying to mkoeppe:
setup.py build_py
should probably use a symlink from/var/tmp...
into~/.sage
as well so that it is easy to test relocation after removing the symlink.
Does not work because sage resolves symlinks when it determines SAGE_ROOT.
comment:8 Changed 7 months ago by
 Dependencies changed from #29039 to #29039, #31409
comment:9 Changed 7 months ago by
 Commit changed from f785d3cde259485597664306e191a21ad0ab9d58 to e56a2a956a96bd70dbb443869697e89b7fb2b58a
Branch pushed to git repo; I updated commit sha1. New commits:
6da7ba2  Add variable TARGETS

b2d4ab8  build/pkgs/{r,rpy2}: Downgrade to optional

8732076  src/sage/repl/ipython_tests.py: Mark R interface tests # optional  rpy2

dbdf084  src/sage/structure/sage_object.pyx: Mark R interface test # optional  rpy2

737b21c  src/sage/stats/r.py: Mark all 2 doctests in this file as # optional  rpy2

4403924  src/sage/interfaces/r.py: Mark all tests # optional  rpy2

e56a2a9  Merge branch 't/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional' into t/31396/relocatable_wheel_version_of_package_sage_conf

comment:10 Changed 7 months ago by
 Commit changed from e56a2a956a96bd70dbb443869697e89b7fb2b58a to a2289744baf90e3dcb288f96ef52d471ab8251bb
Branch pushed to git repo; I updated commit sha1. New commits:
a228974  src/pkgs/sage_confrelocatable/setup.py: Move the SAGE_ROOT directory away after building

comment:11 Changed 7 months ago by
 Commit changed from a2289744baf90e3dcb288f96ef52d471ab8251bb to c260942d3085fa369117746c635b48da5a572489
Branch pushed to git repo; I updated commit sha1. New commits:
0cac965  build/pkgs/ecl: Update to 21.2.1, remove patches included in new version

6a62484  delete all merged upstream patches: MR 210, MR 231

62fac00  Merge branch 't/31336/upgrade_ecl_to_21_2_1' into t/31396/relocatable_wheel_version_of_package_sage_conf

c260942  src/pkgs/sage_confrelocatable/setup.py: Use enabledownloadfromupstreamurl

comment:12 Changed 7 months ago by
 Commit changed from c260942d3085fa369117746c635b48da5a572489 to 4f17a047ec84130462cb7f91f28ed284816cfa74
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
dc5e225  build/pkgs/python3/spkgconfigure.m4: On macOS, if the distutils test fails, try using empty ARCHFLAGS

df17ad4  tox.ini: Add configuration factors for specific homebrew python3.x

914ea58  build/pkgs/python3/spkgconfigure.m4: Get rid of stray AC_SUBST

986739b  m4/sage_check_python_for_venv.m4: Fix reason

0b3e70d  build/pkgs/python3/spkgconfigure.m4: If PYTHON_FOR_VENV is configured to build multiarch extensions, set SAGE_ARCHFLAGS to disable it

372adcb  .github/workflows/tox.yml: Update xcode versions

fc8b676  SAGE_CHECK_PYTHON_FOR_VENV: Rework with less nesting

5ae8044  Merge branch 'u/mkoeppe/accept__usr_bin_python3_from_xcode_12_3_on_macos_10_15__catalina_' of git://trac.sagemath.org/sage into t/31396/relocatable_wheel_version_of_package_sage_conf

c5c73a9  src/pkgs/sage_confrelocatable/tox.ini: Define environments macospython3.8, macospython3.7, macospython3_xcode

4f17a04  src/pkgs/sage_confrelocatable/setup.py: Use enablefatbinary

comment:13 Changed 7 months ago by
 Description modified (diff)
comment:14 Changed 7 months ago by
 Commit changed from 4f17a047ec84130462cb7f91f28ed284816cfa74 to 851435289634de431fc8708e0c4c59a37b578b71
Branch pushed to git repo; I updated commit sha1. New commits:
8514352  src/pkgs/sage_confrelocatable/tox.ini: Run delocatelistdeps

comment:15 Changed 7 months ago by
 Description modified (diff)
comment:16 Changed 7 months ago by
 Commit changed from 851435289634de431fc8708e0c4c59a37b578b71 to 1baf390bc2bc5c29a7bfa6d7c73d31f39a1b5e65
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
c3e4093  Rename SAGE_OPTIONAL_CLEANED_PACKAGES to SAGE_OPTIONAL_UNINSTALLED_PACKAGES

4916415  Merge tag '9.3.beta9' into t/30383/new_package_type__optional_enabled_by_default

a35428e  Merge #30383

e098133  src/pkgs/sage_conf/sage_conf.py.in: Add SAGE_SPKG_WHEELS

5dbd712  src/pkgs/sage_confpypi/setup.py: Use 'configure disablenotebook', run 'make build' so that wheels are built

839c534  src/pkgs/sage_confpypi/setup.py: Fix typo

100ff51  src/pkgs/sage_confpypi/MANIFEST.in: Graft sage_root/build/pkgs/sage_docbuild/src

afa2ffd  Add configure option disablesagelib

a4f75b8  src/pkgs/sage_confpypi: Use configure disablesagelib, update README

1baf390  Merge branch 't/29039/pip_installable_package_sage_bootstrap' into t/31396/relocatable_wheel_version_of_package_sage_conf

comment:17 Changed 6 months ago by
 Commit changed from 1baf390bc2bc5c29a7bfa6d7c73d31f39a1b5e65 to 31bba97d13f86c531eeef0a26383b98fa3671c45
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
f04c134  Merge branch 't/30383/new_package_type__optional_enabled_by_default' into t/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional

27c4bb9  Merge #31409

66c2987  build/pkgs/{sage_sws2rst,sage_docbuild}/installrequires.txt: New

7f8ec6c  build/pkgs/sagelib/src/tox.ini: Add factor sitepackages

0283da5  build/make/Makefile.in: Add wheel, setuptools_wheel to PYTHON_TOOLCHAIN to make sure that PEP 517 packages have a complete build system

f720722  build/pkgs/sagelib/src/tox.ini: Add factor nobuildisolation

c451b31  src/setup.cfg.m4 (install_requires): Add sage_conf

6700223  Merge tag '9.3.rc0' into t/30913/sagelib__add_setup_cfg__install_requires_

ac0f563  Merge branch 't/30913/sagelib__add_setup_cfg__install_requires_' into t/29039/pip_installable_package_sage_bootstrap

31bba97  Merge #29039

comment:18 Changed 6 months ago by
 Cc dimpase added
comment:19 Changed 6 months ago by
 Commit changed from 31bba97d13f86c531eeef0a26383b98fa3671c45 to f5f4a157e3a1630fc789f8f61655e123b6573eb6
Branch pushed to git repo; I updated commit sha1. New commits:
f5f4a15  src/pkgs/sage_confrelocatable/setup.py: Build the wheelhouse, but disablenotebook disabler disablesagelib

comment:20 Changed 6 months ago by
 Description modified (diff)
comment:21 Changed 6 months ago by
 Description modified (diff)
comment:22 Changed 6 months ago by
 Commit changed from f5f4a157e3a1630fc789f8f61655e123b6573eb6 to 87db1d0a371b70040b4d7448f0fadc9b50f18b54
Branch pushed to git repo; I updated commit sha1. New commits:
87db1d0  src/pkgs/sage_confrelocatable/setup.py: Use withmp=gmp instead of broken withoutsystemgmp option

comment:23 Changed 6 months ago by
 Dependencies changed from #29039, #31409 to #29039, #31409, #31521
comment:24 Changed 6 months ago by
 Commit changed from 87db1d0a371b70040b4d7448f0fadc9b50f18b54 to b871e09db5d847470fcf33eab2960554e0787bc1
comment:25 Changed 6 months ago by
 Description modified (diff)
comment:26 Changed 6 months ago by
delocatelistdeps
reports the following library dependencies:
/usr/local/Cellar/gettext/0.21/lib/libintl.8.dylib /usr/local/Cellar/gmp/6.2.1/lib/libgmp.10.dylib /usr/local/Cellar/harfbuzz/2.8.0/lib/libharfbuzz.0.dylib /usr/local/Cellar/libtiff/4.2.0/lib/libtiff.5.dylib /usr/local/Cellar/libxcb/1.14_1/lib/libxcb.1.1.0.dylib /usr/local/Cellar/littlecms2/2.12/lib/liblcms2.2.dylib /usr/local/Cellar/mpfr/4.1.0/lib/libmpfr.6.dylib /usr/local/Cellar/openjpeg/2.4.0/lib/libopenjp2.2.4.0.dylib /usr/local/Cellar/readline/8.1/lib/libreadline.8.1.dylib /usr/local/Cellar/webp/1.2.0/lib/libwebp.7.dylib /usr/local/Cellar/webp/1.2.0/lib/libwebpdemux.2.dylib /usr/local/Cellar/webp/1.2.0/lib/libwebpmux.3.dylib /usr/local/Cellar/zeromq/4.3.3/lib/libzmq.5.dylib /usr/local/Cellar/zlib/1.2.11/lib/libz.1.2.11.dylib @executable_path/../../../../Python3 liba.dylib libb.dylib libc.dylib
comment:27 Changed 6 months ago by
 Commit changed from b871e09db5d847470fcf33eab2960554e0787bc1 to 74a0ae02a05213aa5ec201245aaf3f55e8c5b037
comment:28 Changed 6 months ago by
 Commit changed from 74a0ae02a05213aa5ec201245aaf3f55e8c5b037 to 5eaaa8788437022cb2f070ef12e461e8e9619532
Branch pushed to git repo; I updated commit sha1. New commits:
aa8ff50  src/pkgs/sage_confrelocatable/setup.py: Use withoutsystemreadline

5574aa3  build/pkgs/giac/spkginstall.in [macOS]: Pass withincludedgettext so that it does not link to Homebrew libintl

f71ba56  configure.ac: If disablenotebook and disablesagelib, do not build zeromq etc.

5eaaa87  src/pkgs/sage_confrelocatable/setup.py: Configure libgd without freetype

comment:29 Changed 6 months ago by
/usr/local/Cellar/gettext/0.21/lib/libintl.8.dylib /usr/local/Cellar/harfbuzz/2.8.0/lib/libharfbuzz.0.dylib /usr/local/Cellar/libtiff/4.2.0/lib/libtiff.5.dylib /usr/local/Cellar/libxcb/1.14_1/lib/libxcb.1.1.0.dylib /usr/local/Cellar/littlecms2/2.12/lib/liblcms2.2.dylib /usr/local/Cellar/openjpeg/2.4.0/lib/libopenjp2.2.4.0.dylib /usr/local/Cellar/webp/1.2.0/lib/libwebp.7.dylib /usr/local/Cellar/webp/1.2.0/lib/libwebpdemux.2.dylib /usr/local/Cellar/webp/1.2.0/lib/libwebpmux.3.dylib /usr/local/Cellar/zlib/1.2.11/lib/libz.1.2.11.dylib @executable_path/../../../../Python3 liba.dylib libb.dylib libc.dylib
comment:30 followup: ↓ 36 Changed 6 months ago by
libintl
is still coming in through giac
comment:31 Changed 6 months ago by
 Commit changed from 5eaaa8788437022cb2f070ef12e461e8e9619532 to b7051a99473e4ffa3f2d17c980b619bb45472980
comment:32 followup: ↓ 39 Changed 6 months ago by
The remaining shared libraries from /usr/local
come in through our freetype
(which uses libharfbuzz from /usr/local
), libgd
(which uses libtiff), and pillow
(libtiff and everything else).
We could try not to ship matplotlib
at all in the wheelhouse  it is responsible for pulling in freetype
and pillow
 but it still shares one shared library with some of our packages  libpng
.
comment:33 Changed 6 months ago by
 Commit changed from b7051a99473e4ffa3f2d17c980b619bb45472980 to 41ac551c9b89fb9615e5b52b6164fa0ea67e3a30
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
760b6f0  configure.ac: If disablenotebook and disablesagelib, do not build zeromq etc.

0df51e5  configure.ac (disablenotebook): Disable argon2_cffi

42db661  configure.ac (disablenotebook disablesagelib): Disable ipython

c63ccb0  src/pkgs/sage_confrelocatable/tox.ini (macos): Also use isysroot with g++

1d12382  src/pkgs/sage_confrelocatable/setup.py: Use LIBGD_CONFIGURE='without...almost everything...'

1ef46d0  build/pkgs/giac: Add another patch in patches/autotools

1b0a9ee  build/pkgs/giac: Update to 1.6.0.47p3

41ac551  Merge #31562

comment:34 Changed 6 months ago by
 Dependencies changed from #29039, #31409, #31521 to #29039, #31409, #31521, #31552, #31562
comment:35 Changed 6 months ago by
 Commit changed from 41ac551c9b89fb9615e5b52b6164fa0ea67e3a30 to 00e39560744ad3832aa0614a34b8059a47db5a14
comment:36 in reply to: ↑ 30 Changed 6 months ago by
comment:37 Changed 6 months ago by
 Commit changed from 00e39560744ad3832aa0614a34b8059a47db5a14 to e41932acfaf19f3d19c640d91868696f9b8c5b94
Branch pushed to git repo; I updated commit sha1. New commits:
e41932a  src/pkgs/sage_confrelocatable/tox.ini (macos): Add more SDK paths

comment:38 Changed 6 months ago by
 Work issues set to Refactor through #31567
comment:39 in reply to: ↑ 32 Changed 6 months ago by
Replying to mkoeppe:
The remaining shared libraries from
/usr/local
come in through ourfreetype
(which uses libharfbuzz from/usr/local
)
freetype checks for harfbuzz using pkgconfig, accepts configure withoutharfbuzz
. https://github.com/freetype/freetype/blob/f9f6adb625c48ef15b5d61a3ac1709a068ea95a3/builds/unix/configure.raw#L504
comment:40 Changed 6 months ago by
 Dependencies changed from #29039, #31409, #31521, #31552, #31562 to #29039, #31409, #31521, #31552, #31562, #31567
comment:41 Changed 6 months ago by
 Description modified (diff)
comment:42 Changed 6 months ago by
 Commit changed from e41932acfaf19f3d19c640d91868696f9b8c5b94 to 4ed1bf421731e1cf7d4ad7d1179fa660e5d4364d
Branch pushed to git repo; I updated commit sha1. New commits:
87e2dba  tox.ini: Add localmacosnohomebrew and configuration facors macos{10.14,10.15,11.1 }

0458f23  tox.ini: Add macos{python3_xcode,nohomebrew}python{3.7,3.8 }

39629dd  Merge #31552

9d9c562  Merge #31562

a5f1fc7  Merge #31567

3dd4885  build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no

68c447f  tox.ini (local  bash): Set PS1

4ed1bf4  src/pkgs/sage_confrelocatable: Remove settings that are now set in SAGE_ROOT/tox.ini

comment:43 Changed 6 months ago by
 Work issues Refactor through #31567 deleted
comment:44 Changed 6 months ago by
 Description modified (diff)
comment:45 Changed 6 months ago by
 Description modified (diff)
comment:46 Changed 6 months ago by
 Description modified (diff)
comment:47 Changed 6 months ago by
 Commit changed from 4ed1bf421731e1cf7d4ad7d1179fa660e5d4364d to e4270dd90169c7ba7eb0d9a2c4356dc9c2514941
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
300d054  build/pkgs/pillow/spkginstall.in: Rename environment variable extra_build_ext to PILLOW_BUILD_EXT

8c8d83d  build/pkgs/freetype/spkginstall.in: Pass environment variable FREETYPE_CONFIGURE to configure

12c84a3  tox.ini (macosnohomebrew): Prevent /usr/local from leaking in through boost, freetype, pillow

7966b66  Merge #31409

fadd2b9  Merge #31567

59f41b9  src/pkgs/sage_confrelocatable/setup.py: Resolve SAGE_ROOT

b5edde2  tox.ini (macosnohomebrew): Fix typo in PILLOW_...

e15b152  Merge #31567

dffcdbc  tox.ini (macosnohomebrew): Fix up pillow build by setting ZLIB_ROOT

e4270dd  Merge #31567

comment:48 Changed 6 months ago by
 Commit changed from e4270dd90169c7ba7eb0d9a2c4356dc9c2514941 to 587142f996c7c1457344e960fea682522f00d8e4
Branch pushed to git repo; I updated commit sha1. New commits:
4069297  src/pkgs/sage_confrelocatable/setup.py: Unlink a previous symlink 'sage_root' in the package source tree before setting the new one

3779a25  tox.ini (macosnohomebrew): Disable more libraries to remove dependencies on homebrew

587142f  Merge #31567

comment:49 Changed 6 months ago by
 Commit changed from 587142f996c7c1457344e960fea682522f00d8e4 to c5a823cb524914ab985eaea09f52a87aa93c03cb
Branch pushed to git repo; I updated commit sha1. New commits:
c5a823c  src/pkgs/sage_confrelocatable/setup.py: Make sure that we do not package a copy of the sagelib source tree

comment:50 Changed 6 months ago by
 Description modified (diff)
comment:51 Changed 6 months ago by
 Description modified (diff)
comment:52 Changed 6 months ago by
 Description modified (diff)
comment:53 Changed 6 months ago by
 Cc jhpalmieri slelievre added
comment:54 Changed 6 months ago by
 Status changed from new to needs_review
comment:55 Changed 6 months ago by
 Description modified (diff)
comment:56 Changed 6 months ago by
 Description modified (diff)
comment:57 Changed 6 months ago by
 Status changed from needs_review to needs_work
comment:58 Changed 6 months ago by
 Description modified (diff)
comment:59 Changed 6 months ago by
 Milestone changed from sage9.3 to sage9.4
comment:60 Changed 6 months ago by
 Description modified (diff)
comment:61 Changed 6 months ago by
 Description modified (diff)
comment:62 Changed 6 months ago by
 Dependencies changed from #29039, #31409, #31521, #31552, #31562, #31567 to #29039, #31409, #31521, #31552, #31562, #31567, #29013
comment:63 Changed 6 months ago by
 Description modified (diff)
comment:64 Changed 6 months ago by
 Commit changed from c5a823cb524914ab985eaea09f52a87aa93c03cb to a3ccb60111184d368ecc2a97270b07d158749b3e
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
30926d8  .github/workflows/tox*.yml: Replace ubuntufocali386 by manylinux2_24i686

a06ccf7  Merge #31541

bf6e5f8  build/make/Makefile.in: Change some SAGE_LOCAL to SAGE_VENV

832f96a  build/make/Makefile.in: Rework using tree_... variables

3436ad6  build/make/Makefile.in: Define SAGE_VENV earlier

8cc0b3c  WIP configure withsagevenv

1cfbe2b  build/bin/sagespkg: Change more SAGE_LOCAL to SAGE_INST_LOCAL

2202f84  configure.ac: Finish withsagevenv handling

fa897c6  Merge branch 't/29013/separate_venvs' into t/31396/relocatable_wheel_version_of_package_sage_conf

a3ccb60  src/pkgs/sage_confrelocatable/setup.py: Build the wheels in a venv outside of SAGE_LOCAL

comment:65 Changed 6 months ago by
 Commit changed from a3ccb60111184d368ecc2a97270b07d158749b3e to a295b811b16db09f37aaedccc28d942aaf7fb455
Branch pushed to git repo; I updated commit sha1. New commits:
b604a68  build/make/Makefile.in: Also source sagesrcenvconfig for script packages

92f1a1b  build/pkgs/pynac/spkginstall.in: Build with PYTHON_FOR_VENV

c94af4b  Merge branch 't/29013/separate_venvs' into t/31396/relocatable_wheel_version_of_package_sage_conf

9f1f794  src/pkgs/sage_confrelocatable/setup.py: Include plat_name in the directory name for SAGE_ROOT

d19c5a4  Fixup

5382ec7  src/pkgs/sage_confrelocatable/setup.py: Working version

a295b81  .gitignore: Add more

comment:66 Changed 6 months ago by
 Commit changed from a295b811b16db09f37aaedccc28d942aaf7fb455 to 40492737081faab49c4af8f6445237873f7670a6
comment:67 Changed 6 months ago by
 Description modified (diff)
comment:68 Changed 6 months ago by
 Description modified (diff)
comment:69 Changed 6 months ago by
Unfortunately it turns out that PyPI does not allow @ references in install requires (see https://github.com/pypa/pip/issues/6301)
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/ Invalid value for requires_dist. Error: Can't have direct dependency: 'Cython @ https://github.com/sagemath/sagewheels/releases/download/9.3.rc1/Cython0.29.21cp38cp38macosx_10_15_x86_64.whl'
comment:70 Changed 6 months ago by
 Description modified (diff)
comment:71 Changed 6 months ago by
 Description modified (diff)
comment:72 Changed 6 months ago by
 Description modified (diff)
comment:73 Changed 6 months ago by
 Commit changed from 40492737081faab49c4af8f6445237873f7670a6 to 5871e7cb441c2fc43c71823b15355d28c9397d96
comment:74 Changed 6 months ago by
 Description modified (diff)
comment:75 Changed 6 months ago by
 Commit changed from 5871e7cb441c2fc43c71823b15355d28c9397d96 to 6a3d3807c0029781829b0d50bedecfd5acd38dcd
Branch pushed to git repo; I updated commit sha1. New commits:
7ff26df  git grep l SAGE_SUITESPARSE  xargs sed i .bak s/SAGE_SUITESPARSE_LOCALINSTALL/SAGE_SUITESPARSE_PREFIX/g

52537d1  build/pkgs/cvxopt/spkginstall.in: Actually use the value of SAGE_SUITESPARSE_PREFIX

ed1a089  Merge #31584

be5b7bc  Merge #31567

1fb3a9b  build/pkgs/pynac/spkginstall.in: Back to PYTHON=python3

24ebfb5  src/bin/sage: Only do the check for an incomplete installation when run from the source tree

f331ee7  src/pkgs/sage_confrelocatable/setup.py: Pass CONFIGURED_{CC,CXX} to configure if set

6a3d380  src/pkgs/sage_confrelocatable/setup.py: Package libexec too

comment:76 Changed 6 months ago by
 Description modified (diff)
comment:77 Changed 6 months ago by
 Status changed from needs_work to needs_review
comment:78 Changed 6 months ago by
 Description modified (diff)
comment:79 Changed 6 months ago by
 Commit changed from 6a3d3807c0029781829b0d50bedecfd5acd38dcd to cabb70600d58aaf706166e6dd100b22975ba99c2
Branch pushed to git repo; I updated commit sha1. New commits:
cabb706  src/pkgs/sage_confrelocatable/setup.py: configure withoutsystemboost...

comment:80 Changed 6 months ago by
 Commit changed from cabb70600d58aaf706166e6dd100b22975ba99c2 to 2cb38149fff1211022145aa23a11c3eb10fd9946
Branch pushed to git repo; I updated commit sha1. New commits:
1942363  build/pkgs/pynac/spkginstall.in: Back to PYTHON=python3

9dcded5  src/bin/sage: Only do the check for an incomplete installation when run from the source tree

6e0c19e  sage.misc.package.installed_packages: After SAGE_SPKG_INST, also go through SAGE_VENV_SPKG_INST

12d34ee  Merge #29013

5161d65  build/pkgs/sage_conf/src/sage_conf.py.in: Move SAGE_ROOT, SAGE_LOCAL to beginning of file; only use substitution @prefix@ once

986ca18  build/pkgs/sage_conf/src/sage_conf.py.in: replace subst by replace

6f05cc2  sage_setup.setenv: New, use it in setup.py so that build works outside of sageenv

e7ff393  Merge #31338

2cb3814  src/pkgs/sage_confrelocatable/sage_conf.py.in: Update from src/pkgs/sage_conf/

comment:81 Changed 6 months ago by
 Dependencies changed from #29039, #31409, #31521, #31552, #31562, #31567, #29013 to #29039, #31409, #31521, #31552, #31562, #31567, #29013, #31338
comment:82 Changed 6 months ago by
 Commit changed from 2cb38149fff1211022145aa23a11c3eb10fd9946 to a19f514aca8705e8f6d3340ea3367ea60e7c1ff4
Branch pushed to git repo; I updated commit sha1. New commits:
c5dd7ba  Merge tag '9.3.rc2' into t/29013/public/29013usepy3venv

f0fb082  sage.misc.package.installed_packages: Refactor through new generator _spkg_inst_dirs

d68e861  sage.misc.package.package_manifest: Go through all _spkg_inst_dirs()

54333e5  Merge branch 't/29013/public/29013usepy3venv' into t/31396/relocatable_wheel_version_of_package_sage_conf

a19f514  src/pkgs/sage_conf*/README.rst: Unify the files

comment:83 Changed 6 months ago by
 Commit changed from a19f514aca8705e8f6d3340ea3367ea60e7c1ff4 to 7484dda7d88d41ca84b6f621b4726ae76d89609e
Branch pushed to git repo; I updated commit sha1. New commits:
7484dda  build/pkgs/sagelib/src/README.rst: Link to sageconf on pypi

comment:84 Changed 6 months ago by
 Dependencies changed from #29039, #31409, #31521, #31552, #31562, #31567, #29013, #31338 to #29039, #31562, #31567, #29013, #31338
comment:85 Changed 6 months ago by
 Commit changed from 7484dda7d88d41ca84b6f621b4726ae76d89609e to e523da72c6cb58e87101595e751c26fdbbb208c9
Branch pushed to git repo; I updated commit sha1. New commits:
e523da7  tox.ini (localmacosnohomebrew): Add to Python3.framework/Headers to CPATH

comment:86 Changed 6 months ago by
 Description modified (diff)
comment:87 Changed 6 months ago by
 Dependencies changed from #29039, #31562, #31567, #29013, #31338 to #29039, #31562, #31567, #29013, #31338, #31427
comment:88 Changed 6 months ago by
 Commit changed from e523da72c6cb58e87101595e751c26fdbbb208c9 to c080bbca5a26118d9ae5a5fd4b4a66f43f5c2eeb
Branch pushed to git repo; I updated commit sha1. New commits:
37c9fc0  Makefile: New targets pypisdists, sagewheels

3a8406e  Fixup: Add missing symlinks src/pkgs/sage_conf{pypi,relocatable}/README.rst

82d9f14  build/bin/writedockerfile.sh: ADD some missing files in SAGE_ROOT

7d10bd3  tox.ini (docker), build/bin/writedockerfile.sh: Handle EXTRA_SAGE_PACKAGES

4a24bc5  src/pkgs/sage_confrelocatable/setup.py: Use configure enablebuildasroot

c92b335  build/pkgs/pynac/spkginstall.in: Explicitly use python3 in SAGE_VENV

32c069a  .gitignore: Integrate src/{doc,sage}/.gitignore here so that Docker ignores the listed files too

c080bbc  Merge #31427

comment:89 Changed 6 months ago by
 Commit changed from c080bbca5a26118d9ae5a5fd4b4a66f43f5c2eeb to 22c214e2c09d80075a161b0b308045eef8fa3e7f
comment:90 Changed 6 months ago by
 Description modified (diff)
comment:91 Changed 6 months ago by
 Commit changed from 22c214e2c09d80075a161b0b308045eef8fa3e7f to 79e2ef4ba990356d39bbc01021a5570e48831681
Branch pushed to git repo; I updated commit sha1. New commits:
5ba62ae  .gitignore: Ignore spkg/pkgs/*/{build,dist }

0808863  src/pkgs/sage_confrelocatable/setup.py: Move/copy/delete build trees more carefully

79e2ef4  tox.ini (manylinuxpython3...): Add EXTRA_SAGE_PACKAGES so that system python is accepted

comment:92 Changed 6 months ago by
 Description modified (diff)
comment:93 Changed 6 months ago by
 Commit changed from 79e2ef4ba990356d39bbc01021a5570e48831681 to cc8297ca08d46c736fda5bc2d226326b2ae3da95
Branch pushed to git repo; I updated commit sha1. New commits:
9a3aa38  build/bin/writedockerfile.sh: ADD some missing files in SAGE_ROOT

1b03fa9  src/pkgs/sage_confpypi/setup.py: Change SETENV so that it works in shells with set e

b3b54b7  Merge tag '9.3.rc2' into t/29039/pip_installable_package_sage_bootstrap

ec10bc9  src/pkgs/sage_confpypi/setup.py: Fix for shell behavior on fedora

278c36c  src/pkgs/sage_confpypi/setup.py: Use 'configure enablebuildasroot' so it works in containers too

cc8297c  Merge #29039

comment:94 Changed 6 months ago by
 Commit changed from cc8297ca08d46c736fda5bc2d226326b2ae3da95 to b54e77a8c88b860ac1c5800a0eadbec7385ec98e
comment:95 Changed 6 months ago by
 Dependencies changed from #29039, #31562, #31567, #29013, #31338, #31427 to #29039, #31562, #31567, #29013, #31338, #31427, #28890
comment:96 Changed 6 months ago by
 Commit changed from b54e77a8c88b860ac1c5800a0eadbec7385ec98e to 4dcd1358c6ec1f773356701a5e52329776a653d8
Branch pushed to git repo; I updated commit sha1. New commits:
3f4a8ef  Trac #28890: Pass disablestatic to autoconf packages by default.

8a8efee  Trac #28890: Don't install PARI's static library.

cd39c76  Trac #28890: Don't use sdh_configure to configure zlib; it does not use

9622d83  build/pkgs/zlib/spkginstall.in: Fixup

3d701f8  build/pkgs/zlib/spkginstall.in: Handle errors from configure

56178ab  build/bin/sagedisthelpers (sdh_configure): On Cygwin, do not use disablestatic (until #30814 is done)

4dcd135  Merge #28890

comment:97 Changed 6 months ago by
 Commit changed from 4dcd1358c6ec1f773356701a5e52329776a653d8 to 6f7819ee24e648867fe91f1e9f2536f27ff10133
comment:98 Changed 5 months ago by
 Commit changed from 6f7819ee24e648867fe91f1e9f2536f27ff10133 to 9ab6db9a51719cc2cfbf0f244143d6a59005f1fb
comment:99 Changed 4 months ago by
 Dependencies changed from #29039, #31562, #31567, #29013, #31338, #31427, #28890 to #29039, #31338, #28890
comment:100 Changed 4 months ago by
 Dependencies changed from #29039, #31338, #28890 to #29039, #31338, #29617
comment:101 Changed 4 months ago by
 Commit changed from 9ab6db9a51719cc2cfbf0f244143d6a59005f1fb to ce5104854ea85eed635dfe5ce81da7f55fcd46e1
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
133aab7  Trac #29617: move MAXIMA_FAS to sage_conf.

9b595c0  Trac #29617: modify Kenzo interface to support system ECL.

b4ae35e  Trac #29617: treat empty MAXIMA_FAS and KENZO_FAS as None.

2d40f36  Trac #29617: upgrade kenzo to v1.1.10.

2078fa6  Merge tag '9.4.beta0' into t/31593/configure__paths_within__sage_local___prefix__for_sage_conf

5cbe423  Merge #31593

be48dd8  Merge #29617

b4b3ec8  Merge tag '9.4.beta0' into t/29039/pip_installable_package_sage_bootstrap

e1d2df0  Merge tag '9.4.beta1' into t/29039/pip_installable_package_sage_bootstrap

ce51048  Merge #29039

comment:102 Changed 4 months ago by
 Commit changed from ce5104854ea85eed635dfe5ce81da7f55fcd46e1 to 52d669df18cf3b2d4396094bcd4ab99049fb7d33
Branch pushed to git repo; I updated commit sha1. New commits:
7e8a7a8  Makefile (sagewheels): Use SETUPTOOLS_USE_DISTUTILS=local

801e792  src/pkgs/sage_confrelocatable/setup.py: Use wheel tags in build directory name too

52d669d  src/pkgs/sage_confrelocatable/setup.py: Do not use removed option withoutsystemboost

comment:103 Changed 4 months ago by
 Commit changed from 52d669df18cf3b2d4396094bcd4ab99049fb7d33 to 55122f3fbfc95f2ef58786b65ef17d8dc6e52037
Branch pushed to git repo; I updated commit sha1. New commits:
55122f3  build/pkgs/pynac/spkginstall.in: Work around Trac #31920

comment:104 Changed 4 months ago by
 Commit changed from 55122f3fbfc95f2ef58786b65ef17d8dc6e52037 to ff58be39968b29d031676ded20286080d2a05feb
Branch pushed to git repo; I updated commit sha1. New commits:
ff58be3  Makefile (sagewheels): Fix wheel location in final message

comment:105 Changed 4 months ago by
Installation with /usr/bin/python3 m pip install I user https://github.com/sagemath/sagewheels/releases/download/9.4.beta1/sage_conf9.4b1cp38cp38macosx_11_0_x86_64.whl https://github.com/sagemath/sagewheels/releases/download/9.4.beta1/sagemath_standard9.4b1cp38cp38macosx_11_0_x86_64.whl
fails with:
Collecting argon2cffi (from notebook>=4.4.1>widgetsnbextension~=3.5.0>ipywidgets>=7.5.1>sagemathstandard==9.4b1) Downloading https://files.pythonhosted.org/packages/74/fd/d78e003a79c453e8454197092fce9d1c6099445b7e7da0b04eb4fe1dbab7/argon2cffi20.1.0.tar.gz (1.8MB) [...] Failed to build argon2cffi
comment:106 Changed 4 months ago by
#29039 is listed as a dependency, is it an error?
comment:107 Changed 4 months ago by
No, that's correct
comment:108 Changed 3 months ago by
 Commit changed from ff58be39968b29d031676ded20286080d2a05feb to 1c35e0f48ac8edca91037980292f732692490e89
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
76023cc  .gitignore: Ignore spkg/pkgs/*/{build,dist }

8591334  src/pkgs/sage_confrelocatable/setup.py: Move/copy/delete build trees more carefully

adbbdc1  tox.ini (manylinuxpython3...): Add EXTRA_SAGE_PACKAGES so that system python is accepted

6a6a66a  Makefile (sagewheels): Use SETUPTOOLS_USE_DISTUTILS=local

64ff2e7  src/pkgs/sage_confrelocatable/setup.py: Use wheel tags in build directory name too

40050a2  src/pkgs/sage_confrelocatable/setup.py: Do not use removed option withoutsystemboost

a2aa592  build/pkgs/pynac/spkginstall.in: Work around Trac #31920

52ecb27  Makefile (sagewheels): Fix wheel location in final message

84637dd  Move sprc/pkgs/sage_confrelocatable to pkgs/sageconf_relocatable

1c35e0f  Put pkgs/sageconf in the root

comment:109 Changed 3 months ago by
 Dependencies changed from #29039, #31338, #29617 to #29039
comment:110 Changed 3 months ago by
 Commit changed from 1c35e0f48ac8edca91037980292f732692490e89 to 59d5a9f99135a2a7965d6c6d041f90aea92d1d6f
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
3b863d7  configure.ac: If disablenotebook and disablesagelib, do not build zeromq etc.

bfab056  tox.ini (local  bash): Set PS1

1b41f14  configure.ac: Add configure option disabledoc

e2522ff  tox.ini (docker), build/bin/writedockerfile.sh: Handle EXTRA_SAGE_PACKAGES

7507b22  build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no

449fe19  build/pkgs/pynac/spkginstall.in: Explicitly use python3 in SAGE_VENV

5d24f9d  build/bin/sagespkg: Also source sagesrcenvconfig so that SAGE_VENV is available

2dcbc90  src/pkgs/sage_confrelocatable: New

3bf0a49  Makefile: New targets pypisdists, sagewheels

59d5a9f  Move sprc/pkgs/sage_confrelocatable to pkgs/sageconf_relocatable

comment:111 Changed 3 months ago by
 Commit changed from 59d5a9f99135a2a7965d6c6d041f90aea92d1d6f to 6b7bce6f59de8679622082d65feae52945f0cb8a
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
b5267e8  configure.ac: Add configure option disabledoc

4da52e1  tox.ini (local  bash): Set PS1

f7bfefd  tox.ini (docker), build/bin/writedockerfile.sh: Handle EXTRA_SAGE_PACKAGES

92db7a5  build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no

e914412  build/pkgs/pynac/spkginstall.in: Explicitly use python3 in SAGE_VENV

a879f6d  build/bin/sagespkg: Also source sagesrcenvconfig so that SAGE_VENV is available

691512c  src/pkgs/sage_confrelocatable: New

99cdf97  Makefile: New target sagewheels

6b7bce6  Move sprc/pkgs/sage_confrelocatable to pkgs/sageconf_relocatable

comment:112 Changed 3 months ago by
 Commit changed from 6b7bce6f59de8679622082d65feae52945f0cb8a to d9554e3422f62a88d3e87fa7449321fa0bdd3664
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
38a7040  configure.ac: If disablenotebook and disablesagelib, do not build zeromq etc.

f8fc4a7  configure.ac: Add configure option disabledoc

c830393  tox.ini (local  bash): Set PS1

aafb323  tox.ini (docker), build/bin/writedockerfile.sh: Handle EXTRA_SAGE_PACKAGES

c2ac2e8  build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no

ac6fcde  build/pkgs/pynac/spkginstall.in: Explicitly use python3 in SAGE_VENV

703c81a  build/bin/sagespkg: Also source sagesrcenvconfig so that SAGE_VENV is available

787d8d3  src/pkgs/sage_confrelocatable: New

a2861aa  Makefile: New target sagewheels

d9554e3  Move sprc/pkgs/sage_confrelocatable to pkgs/sageconf_relocatable

comment:113 Changed 3 months ago by
 Description modified (diff)
comment:114 Changed 3 months ago by
 Commit changed from d9554e3422f62a88d3e87fa7449321fa0bdd3664 to 76f53c74420a32c972b90096432ba37fe8e15d9f
Branch pushed to git repo; I updated commit sha1. New commits:
44b1e8d  Makefile (pypisdists): Add dependency on sage_setup

98cded3  Merge branch 't/29039/pip_installable_package_sage_bootstrap' into t/31396/relocatable_wheel_version_of_package_sage_conf

76f53c7  Makefile (sagewheels), pkgs/sageconf_relocatable/setup.py: Use 'pip wheel' instead of setup.py bdist_wheel; exclude all embedded trees; improve hints and output

comment:115 Changed 3 months ago by
OK, this works, but can an optional package not be supported on 32bit Linux (nor on FreeBSD, which is another platform Sage should work on)
And, how about arm64?
comment:116 Changed 2 months ago by
 Commit changed from 76f53c74420a32c972b90096432ba37fe8e15d9f to 744b58ec2c143cfc43bb31f1e329026f826f8262
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
569b188  Merge #29847

35b8b74  Merge tag '9.4.beta5' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup

773ef29  Merge #32073

9aa1f10  Merge #29585

8c5ee65  build/pkgs/pip/installrequires.txt: Update lower bound

ed5f9f0  Merge tag '9.4.beta6' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

6b1b222  Merge #32046

cfd0c25  Merge #29847

40177f1  Merge #29039

744b58e  Update for changed source layout

comment:117 Changed 2 months ago by
 Milestone changed from sage9.4 to sage9.5
comment:118 Changed 6 weeks ago by
 Commit changed from 744b58ec2c143cfc43bb31f1e329026f826f8262 to bab6c0c6c4360fd391d0a63ae3d29c1c9b50c909
Branch pushed to git repo; I updated commit sha1. New commits:
85d1d27  Merge tag '9.4.rc1' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

050f8df  build/pkgs/numpy/spkginstall.in: Use sdh_pip_install, sdh_prefix_args

a41b507  Merge branch 't/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup

e6e8ced  Merge #29847

d69c496  build/pkgs/ptyprocess: pin version to 0.5.1

7f13d04  Merge #32371

bab6c0c  Merge #29039

comment:119 Changed 6 weeks ago by
 Description modified (diff)
comment:120 Changed 6 weeks ago by
 Commit changed from bab6c0c6c4360fd391d0a63ae3d29c1c9b50c909 to e4b1a959b399e81202f106740f6ce314fc16290e
Branch pushed to git repo; I updated commit sha1. New commits:
290fa2f  build/pkgs/sagelib/src: Restore lost symlink

478ffcd  Merge #32046

b011389  Merge branch 't/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup' into t/29039/pip_installable_package_sage_bootstrap

e4b1a95  Merge #29039

Branch pushed to git repo; I updated commit sha1. New commits:
Fix up package_data
Closer to working