relocatable wheel version of package sage_conf

[mkoeppe]

This version of sage_conf is for making a wheel that packages the precompiled non-Python bits of the Sage distribution, making SAGE_ROOT (and thus SAGE_LOCAL=$SAGE_ROOT/local) relocatable using Marc Culler's symbolic link surgery (the method proposed in #31076, using SAGE_ROOT=/var/tmp/sage-... and a symlink).

The sage script invokes sage-config to determine SAGE_ROOT and SAGE_LOCAL. In the version of sage-config supplied by this version of sage_conf, we ensure that the symlink from /var/tmp/sage-.... to the actual install location is set.

We also refactor sage_conf to make it easier to maintain variants of it, both the now 3 in-tree variants and downstream variants:

• The configure-generated configuration file is now pkgs/sage-conf/_sage_conf/_conf.py
• The file pkgs/sage-conf/sage_conf.py is now static

So far, a wheel has been built for XCode python 3.8 on macOS 10.15. The wheel is 670MB in size, an order of magnitude above the standard file size limit on PyPI; but a file size limit increase (requested in ​https://github.com/pypa/pypi-support/issues/985) has kindly been granted by the PyPI team.

The wheel declares dependencies (install-requires) to all Python packages in the Sage distribution that have extension modules. The dependencies are specific using @ to URLs on ​https://github.com/sagemath/sage-wheels/releases/tag/9.5.rc2, where I have uploaded the binary wheels. Unfortunately, by PyPI policy, such @ references are not allowed for packages on PyPI. Hence, the pip invocation needs to use a URL.

(See also ​https://twitter.com/mkoeppe_math/status/1378860285537054723)

Instructions for testing:

1. Make sure that the bin directory of the user installation scheme is in PATH. For example, on macOS:

   $ export PATH=$HOME/Library/Python/3.8/bin:/usr/bin:/bin:/usr/sbin:/sbin
   

2. Install the wheel in the user scheme:

   $ /usr/bin/python3 -m pip install -U --user  https://github.com/sagemath/sage-wheels/releases/download/9.3.rc1/sagemath_standard-9.3rc1-cp38-cp38-macosx_10_15_x86_64.whl
   

3. Run the installed sage.

   $ sage
   

Procedure for building the sage_conf wheel and compatible wheels:

For building wheels on macOS Catalina that uses XCode python 3.8:

$ SKIP_CONFIGURE=1 tox -e local-macos-10.15-nohomebrew-python3_xcode -- bash
(tox -e ...) $ (cd src/pkgs/sage_conf-relocatable/ && python3 setup.py bdist_wheel --plat-name macosx_10_15_x86_64)

or

tox -e local-macos-10.15-nohomebrew-python3_xcode -- SAGE_CHECK=no sage-wheels

This creates (uploaded to ​https://github.com/sagemath/sage-wheels/releases/tag/9.5.rc2)

• src/pkgs/sage_conf-relocatable/dist/sage_conf-9.3rc1-cp38-cp38-macosx_10_15_x86_64.whl
• src/pkgs/sage_conf-relocatable/sage_root/venv-cpython-38-darwin-macosx_10_15_x86_64/var/lib/sage/wheels/*.whl

Build for macOS Big Sur (requires a machine running Big Sur or Monterey) (uploaded to ​https://github.com/sagemath/sage-wheels/releases/tag/9.5.rc2)

tox -e local-macos-11.1-nohomebrew-python3_xcode -- SAGE_CHECK=no sage-wheels

Build for macOS Monterey (requires a machine running Monterey)

tox -e local-macos-12.1-nohomebrew-python3_xcode -- SAGE_CHECK=no sage-wheels

Build for Linux (does not work yet):

$ TARGETS_PRE=Makefile tox -e docker-manylinux-2_24-minimal-python3.9 -- sage-wheels

Follow-up steps:

• also build a sagemath-standard wheel
• build wheels for manylinux using tox -e docker-manylinux....
  • wheel is much too big: 2.3GB
  • auditwheel complains: RuntimeError: Invalid binary wheel, found the following shared library/libraries in purelib folder:; so we should replace our current abuse of package data (install_data) by platlib (install_lib). ​https://github.com/pypa/packaging-problems/issues/542#issuecomment-912838470: subclass build_ext instead of build_py
• disable building the dependencies of the sphinx packages; remove unneeded static libraries; remove duplicated/triplicated shared libraries if possible -- for example by storing a tar file sage_root.tar (with symlinks) instead of including the sage root as a directory - ​https://docs.python.org/3/library/tarfile.html#tarfile.open
• #31602: split out sage_root from sage_conf and use install-requires in the sage_conf sdist/wheel with @ links to GH releases to the sage_root wheel and the built non-any wheels.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​952fb9a	Fix up package_data
​9573b7c	Closer to working

[mkoeppe]

On macOS, some shared libraries from homebrew still leak into the build:

find src/pkgs/sage_conf-relocatable/.tox/python/lib/python3.9/site-packages/sage_root -name "*.so" -o -name "*.dylib" | xargs otool -L
libgiac.dylib:
/usr/local/opt/gettext/lib/libintl.8.dylib
R/library/cluster/libs/*.dylib:
	/usr/local/opt/gettext/lib/libintl.8.dylib (compatibility version 11.0.0, current version 11.0.0)
	/usr/local/opt/pcre/lib/libpcre.1.dylib (compatibility version 4.0.0, current version 4.12.0)
	/usr/local/opt/xz/lib/liblzma.5.dylib (compatibility version 8.0.0, current version 8.5.0)

Definitely should disable the R build (#30383).

Also check whether setting CC=gcc -sysroot.... helps getting rid of /usr/local leaking in.

Also need to add --enable-fat-binary.

setup.py build_py should probably use a symlink from /var/tmp... into ~/.sage as well so that it is easy to test relocation after removing the symlink. Also copytree from sage_root_source should be changed so it overwrites (updates) the existing contents of SAGE_ROOT.

the tox test should include the test with otool for self-containedness.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f785d3c

src/pkgs/sage_conf-relocatable/tox.ini: Add environment macos-10.15

[mkoeppe]

Using -isysroot helps but giac configuration needs more work - #31403

[mkoeppe]

Replying to mkoeppe:

setup.py build_py should probably use a symlink from /var/tmp... into ~/.sage as well so that it is easy to test relocation after removing the symlink.

Does not work because sage resolves symlinks when it determines SAGE_ROOT.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6da7ba2	Add variable TARGETS
​b2d4ab8	build/pkgs/{r,rpy2}: Downgrade to optional
​8732076	src/sage/repl/ipython_tests.py: Mark R interface tests # optional - rpy2
​dbdf084	src/sage/structure/sage_object.pyx: Mark R interface test # optional - rpy2
​737b21c	src/sage/stats/r.py: Mark all 2 doctests in this file as # optional - rpy2
​4403924	src/sage/interfaces/r.py: Mark all tests # optional - rpy2
​e56a2a9	Merge branch 't/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional' into t/31396/relocatable_wheel_version_of_package_sage_conf

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a228974

src/pkgs/sage_conf-relocatable/setup.py: Move the SAGE_ROOT directory away after building

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0cac965	build/pkgs/ecl: Update to 21.2.1, remove patches included in new version
​6a62484	delete all merged upstream patches: MR 210, MR 231
​62fac00	Merge branch 't/31336/upgrade_ecl_to_21_2_1' into t/31396/relocatable_wheel_version_of_package_sage_conf
​c260942	src/pkgs/sage_conf-relocatable/setup.py: Use --enable-download-from-upstream-url

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​dc5e225	build/pkgs/python3/spkg-configure.m4: On macOS, if the distutils test fails, try using empty ARCHFLAGS
​df17ad4	tox.ini: Add configuration factors for specific homebrew python3.x
​914ea58	build/pkgs/python3/spkg-configure.m4: Get rid of stray AC_SUBST
​986739b	m4/sage_check_python_for_venv.m4: Fix reason
​0b3e70d	build/pkgs/python3/spkg-configure.m4: If PYTHON_FOR_VENV is configured to build multiarch extensions, set SAGE_ARCHFLAGS to disable it
​372adcb	.github/workflows/tox.yml: Update xcode versions
​fc8b676	SAGE_CHECK_PYTHON_FOR_VENV: Rework with less nesting
​5ae8044	Merge branch 'u/mkoeppe/accept__usr_bin_python3_from_xcode_12_3_on_macos_10_15__catalina_' of git://trac.sagemath.org/sage into t/31396/relocatable_wheel_version_of_package_sage_conf
​c5c73a9	src/pkgs/sage_conf-relocatable/tox.ini: Define environments macos-python3.8, macos-python3.7, macos-python3_xcode
​4f17a04	src/pkgs/sage_conf-relocatable/setup.py: Use --enable-fat-binary

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8514352

src/pkgs/sage_conf-relocatable/tox.ini: Run delocate-listdeps

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​c3e4093	Rename SAGE_OPTIONAL_CLEANED_PACKAGES to SAGE_OPTIONAL_UNINSTALLED_PACKAGES
​4916415	Merge tag '9.3.beta9' into t/30383/new_package_type__optional_enabled_by_default
​a35428e	Merge #30383
​e098133	src/pkgs/sage_conf/sage_conf.py.in: Add SAGE_SPKG_WHEELS
​5dbd712	src/pkgs/sage_conf-pypi/setup.py: Use 'configure --disable-notebook', run 'make build' so that wheels are built
​839c534	src/pkgs/sage_conf-pypi/setup.py: Fix typo
​100ff51	src/pkgs/sage_conf-pypi/MANIFEST.in: Graft sage_root/build/pkgs/sage_docbuild/src
​afa2ffd	Add configure option --disable-sagelib
​a4f75b8	src/pkgs/sage_conf-pypi: Use configure --disable-sagelib, update README
​1baf390	Merge branch 't/29039/pip_installable_package_sage_bootstrap' into t/31396/relocatable_wheel_version_of_package_sage_conf

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​f04c134	Merge branch 't/30383/new_package_type__optional_enabled_by_default' into t/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional
​27c4bb9	Merge #31409
​66c2987	build/pkgs/{sage_sws2rst,sage_docbuild}/install-requires.txt: New
​7f8ec6c	build/pkgs/sagelib/src/tox.ini: Add factor sitepackages
​0283da5	build/make/Makefile.in: Add wheel, setuptools_wheel to PYTHON_TOOLCHAIN to make sure that PEP 517 packages have a complete build system
​f720722	build/pkgs/sagelib/src/tox.ini: Add factor nobuildisolation
​c451b31	src/setup.cfg.m4 (install_requires): Add sage_conf
​6700223	Merge tag '9.3.rc0' into t/30913/sagelib__add_setup_cfg__install_requires_
​ac0f563	Merge branch 't/30913/sagelib__add_setup_cfg__install_requires_' into t/29039/pip_installable_package_sage_bootstrap
​31bba97	Merge #29039

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f5f4a15

src/pkgs/sage_conf-relocatable/setup.py: Build the wheelhouse, but --disable-notebook --disable-r --disable-sagelib

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​87db1d0

src/pkgs/sage_conf-relocatable/setup.py: Use --with-mp=gmp instead of broken --without-system-gmp option

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​128e8ef	Revert "do not allow numpy intrinsics when building fat binary"
​b871e09	Merge #31521

[mkoeppe]

delocate-listdeps reports the following library dependencies:

/usr/local/Cellar/gettext/0.21/lib/libintl.8.dylib
/usr/local/Cellar/gmp/6.2.1/lib/libgmp.10.dylib
/usr/local/Cellar/harfbuzz/2.8.0/lib/libharfbuzz.0.dylib
/usr/local/Cellar/libtiff/4.2.0/lib/libtiff.5.dylib
/usr/local/Cellar/libxcb/1.14_1/lib/libxcb.1.1.0.dylib
/usr/local/Cellar/little-cms2/2.12/lib/liblcms2.2.dylib
/usr/local/Cellar/mpfr/4.1.0/lib/libmpfr.6.dylib
/usr/local/Cellar/openjpeg/2.4.0/lib/libopenjp2.2.4.0.dylib
/usr/local/Cellar/readline/8.1/lib/libreadline.8.1.dylib
/usr/local/Cellar/webp/1.2.0/lib/libwebp.7.dylib
/usr/local/Cellar/webp/1.2.0/lib/libwebpdemux.2.dylib
/usr/local/Cellar/webp/1.2.0/lib/libwebpmux.3.dylib
/usr/local/Cellar/zeromq/4.3.3/lib/libzmq.5.dylib
/usr/local/Cellar/zlib/1.2.11/lib/libz.1.2.11.dylib
@executable_path/../../../../Python3
liba.dylib
libb.dylib
libc.dylib

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8b5b273	build/pkgs/singular/checksums.ini: Use 4.2.0p1+2021-03-24+sage-2
​f71b013	Merge #31552
​74a0ae0	src/pkgs/sage_conf-relocatable/setup.py: Do not package up $SAGE_LOCAL/var/tmp

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​aa8ff50	src/pkgs/sage_conf-relocatable/setup.py: Use --without-system-readline
​5574aa3	build/pkgs/giac/spkg-install.in [macOS]: Pass --with-included-gettext so that it does not link to Homebrew libintl
​f71ba56	configure.ac: If --disable-notebook and --disable-sagelib, do not build zeromq etc.
​5eaaa87	src/pkgs/sage_conf-relocatable/setup.py: Configure libgd without freetype

[mkoeppe]

/usr/local/Cellar/gettext/0.21/lib/libintl.8.dylib
/usr/local/Cellar/harfbuzz/2.8.0/lib/libharfbuzz.0.dylib
/usr/local/Cellar/libtiff/4.2.0/lib/libtiff.5.dylib
/usr/local/Cellar/libxcb/1.14_1/lib/libxcb.1.1.0.dylib
/usr/local/Cellar/little-cms2/2.12/lib/liblcms2.2.dylib
/usr/local/Cellar/openjpeg/2.4.0/lib/libopenjp2.2.4.0.dylib
/usr/local/Cellar/webp/1.2.0/lib/libwebp.7.dylib
/usr/local/Cellar/webp/1.2.0/lib/libwebpdemux.2.dylib
/usr/local/Cellar/webp/1.2.0/lib/libwebpmux.3.dylib
/usr/local/Cellar/zlib/1.2.11/lib/libz.1.2.11.dylib
@executable_path/../../../../Python3
liba.dylib
libb.dylib
libc.dylib

[mkoeppe]

libintl is still coming in through giac

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0ccba0c	configure.ac (--disable-notebook): Disable argon2_cffi
​b7051a9	configure.ac (--disable-notebook --disable-sagelib): Disable ipython

[mkoeppe]

The remaining shared libraries from /usr/local come in through our freetype (which uses libharfbuzz from /usr/local), libgd (which uses libtiff), and pillow (libtiff and everything else).

We could try not to ship matplotlib at all in the wheelhouse - it is responsible for pulling in freetype and pillow - but it still shares one shared library with some of our packages - libpng.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​760b6f0	configure.ac: If --disable-notebook and --disable-sagelib, do not build zeromq etc.
​0df51e5	configure.ac (--disable-notebook): Disable argon2_cffi
​42db661	configure.ac (--disable-notebook --disable-sagelib): Disable ipython
​c63ccb0	src/pkgs/sage_conf-relocatable/tox.ini (macos): Also use -isysroot with g++
​1d12382	src/pkgs/sage_conf-relocatable/setup.py: Use LIBGD_CONFIGURE='--without-...almost everything...'
​1ef46d0	build/pkgs/giac: Add another patch in patches/autotools
​1b0a9ee	build/pkgs/giac: Update to 1.6.0.47p3
​41ac551	Merge #31562

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​89f61d2	build/pkgs/giac: Update to 1.6.0.47p3
​00e3956	Merge #31562

[mkoeppe]

Replying to mkoeppe:

libintl is still coming in through giac

Solved now via #31562.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e41932a

src/pkgs/sage_conf-relocatable/tox.ini (macos): Add more SDK paths

[mkoeppe]

Replying to mkoeppe:

The remaining shared libraries from /usr/local come in through our freetype (which uses libharfbuzz from /usr/local)

freetype checks for harfbuzz using pkg-config, accepts configure --without-harfbuzz. ​https://github.com/freetype/freetype/blob/f9f6adb625c48ef15b5d61a3ac1709a068ea95a3/builds/unix/configure.raw#L504

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​87e2dba	tox.ini: Add local-macos-nohomebrew and configuration facors macos-{10.14,10.15,11.1}
​0458f23	tox.ini: Add macos-{python3_xcode,nohomebrew}-python{3.7,3.8}
​39629dd	Merge #31552
​9d9c562	Merge #31562
​a5f1fc7	Merge #31567
​3dd4885	build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no
​68c447f	tox.ini (local -- bash): Set PS1
​4ed1bf4	src/pkgs/sage_conf-relocatable: Remove settings that are now set in SAGE_ROOT/tox.ini

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​300d054	build/pkgs/pillow/spkg-install.in: Rename environment variable extra_build_ext to PILLOW_BUILD_EXT
​8c8d83d	build/pkgs/freetype/spkg-install.in: Pass environment variable FREETYPE_CONFIGURE to configure
​12c84a3	tox.ini (macos-nohomebrew): Prevent /usr/local from leaking in through boost, freetype, pillow
​7966b66	Merge #31409
​fadd2b9	Merge #31567
​59f41b9	src/pkgs/sage_conf-relocatable/setup.py: Resolve SAGE_ROOT
​b5edde2	tox.ini (macos-nohomebrew): Fix typo in PILLOW_...
​e15b152	Merge #31567
​dffcdbc	tox.ini (macos-nohomebrew): Fix up pillow build by setting ZLIB_ROOT
​e4270dd	Merge #31567

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4069297	src/pkgs/sage_conf-relocatable/setup.py: Unlink a previous symlink 'sage_root' in the package source tree before setting the new one
​3779a25	tox.ini (macos-nohomebrew): Disable more libraries to remove dependencies on homebrew
​587142f	Merge #31567

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c5a823c

src/pkgs/sage_conf-relocatable/setup.py: Make sure that we do not package a copy of the sagelib source tree

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​30926d8	.github/workflows/tox*.yml: Replace ubuntu-focal-i386 by manylinux-2_24-i686
​a06ccf7	Merge #31541
​bf6e5f8	build/make/Makefile.in: Change some SAGE_LOCAL to SAGE_VENV
​832f96a	build/make/Makefile.in: Rework using tree_... variables
​3436ad6	build/make/Makefile.in: Define SAGE_VENV earlier
​8cc0b3c	WIP configure --with-sage-venv
​1cfbe2b	build/bin/sage-spkg: Change more SAGE_LOCAL to SAGE_INST_LOCAL
​2202f84	configure.ac: Finish --with-sage-venv handling
​fa897c6	Merge branch 't/29013/separate_venvs' into t/31396/relocatable_wheel_version_of_package_sage_conf
​a3ccb60	src/pkgs/sage_conf-relocatable/setup.py: Build the wheels in a venv outside of SAGE_LOCAL

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b604a68	build/make/Makefile.in: Also source sage-src-env-config for script packages
​92f1a1b	build/pkgs/pynac/spkg-install.in: Build with PYTHON_FOR_VENV
​c94af4b	Merge branch 't/29013/separate_venvs' into t/31396/relocatable_wheel_version_of_package_sage_conf
​9f1f794	src/pkgs/sage_conf-relocatable/setup.py: Include plat_name in the directory name for SAGE_ROOT
​d19c5a4	Fixup
​5382ec7	src/pkgs/sage_conf-relocatable/setup.py: Working version
​a295b81	.gitignore: Add more

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e4e6a1e	src/pkgs/sage_conf-relocatable/setup.py: Fixup
​323dcf1	src/pkgs/sage_conf-relocatable/setup.py: Add first install_requires @ GH
​4049273	src/pkgs/sage_conf-relocatable/setup.py: Add install_requires for all built binary wheels

[mkoeppe]

Unfortunately it turns out that PyPI does not allow @ references in install requires (see ​https://github.com/pypa/pip/issues/6301)

HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Invalid value for requires_dist. Error: Can't have direct dependency: 'Cython @ https://github.com/sagemath/sage-wheels/releases/download/9.3.rc1/Cython-0.29.21-cp38-cp38-macosx_10_15_x86_64.whl'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​916bda8	configure.ac: Add configure option --disable-doc
​5871e7c	Have sagemath_standard wheel depend on sage_conf wheel @ GH

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7ff26df	git grep -l SAGE_SUITESPARSE | xargs sed -i .bak s/SAGE_SUITESPARSE_LOCALINSTALL/SAGE_SUITESPARSE_PREFIX/g
​52537d1	build/pkgs/cvxopt/spkg-install.in: Actually use the value of SAGE_SUITESPARSE_PREFIX
​ed1a089	Merge #31584
​be5b7bc	Merge #31567
​1fb3a9b	build/pkgs/pynac/spkg-install.in: Back to PYTHON=python3
​24ebfb5	src/bin/sage: Only do the check for an incomplete installation when run from the source tree
​f331ee7	src/pkgs/sage_conf-relocatable/setup.py: Pass CONFIGURED_{CC,CXX} to configure if set
​6a3d380	src/pkgs/sage_conf-relocatable/setup.py: Package libexec too

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cabb706

src/pkgs/sage_conf-relocatable/setup.py: configure --without-system-boost...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1942363	build/pkgs/pynac/spkg-install.in: Back to PYTHON=python3
​9dcded5	src/bin/sage: Only do the check for an incomplete installation when run from the source tree
​6e0c19e	sage.misc.package.installed_packages: After SAGE_SPKG_INST, also go through SAGE_VENV_SPKG_INST
​12d34ee	Merge #29013
​5161d65	build/pkgs/sage_conf/src/sage_conf.py.in: Move SAGE_ROOT, SAGE_LOCAL to beginning of file; only use substitution @prefix@ once
​986ca18	build/pkgs/sage_conf/src/sage_conf.py.in: replace subst by replace
​6f05cc2	sage_setup.setenv: New, use it in setup.py so that build works outside of sage-env
​e7ff393	Merge #31338
​2cb3814	src/pkgs/sage_conf-relocatable/sage_conf.py.in: Update from src/pkgs/sage_conf/

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c5dd7ba	Merge tag '9.3.rc2' into t/29013/public/29013-use-py3-venv
​f0fb082	sage.misc.package.installed_packages: Refactor through new generator _spkg_inst_dirs
​d68e861	sage.misc.package.package_manifest: Go through all _spkg_inst_dirs()
​54333e5	Merge branch 't/29013/public/29013-use-py3-venv' into t/31396/relocatable_wheel_version_of_package_sage_conf
​a19f514	src/pkgs/sage_conf*/README.rst: Unify the files

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7484dda

build/pkgs/sagelib/src/README.rst: Link to sage-conf on pypi

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e523da7

tox.ini (local-macos-nohomebrew): Add to Python3.framework/Headers to CPATH

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​37c9fc0	Makefile: New targets pypi-sdists, sage-wheels
​3a8406e	Fixup: Add missing symlinks src/pkgs/sage_conf-{pypi,relocatable}/README.rst
​82d9f14	build/bin/write-dockerfile.sh: ADD some missing files in SAGE_ROOT
​7d10bd3	tox.ini (docker), build/bin/write-dockerfile.sh: Handle EXTRA_SAGE_PACKAGES
​4a24bc5	src/pkgs/sage_conf-relocatable/setup.py: Use configure --enable-build-as-root
​c92b335	build/pkgs/pynac/spkg-install.in: Explicitly use python3 in SAGE_VENV
​32c069a	.gitignore: Integrate src/{doc,sage}/.gitignore here so that Docker ignores the listed files too
​c080bbc	Merge #31427

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​927ce20	build/bin/sage-spkg: Also source sage-src-env-config so that SAGE_VENV is available
​22c214e	src/pkgs/sage_conf-relocatable/setup.py: Use configure --without-system-zeromq

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5ba62ae	.gitignore: Ignore spkg/pkgs/*/{build,dist}
​0808863	src/pkgs/sage_conf-relocatable/setup.py: Move/copy/delete build trees more carefully
​79e2ef4	tox.ini (manylinux-python3...): Add EXTRA_SAGE_PACKAGES so that system python is accepted

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9a3aa38	build/bin/write-dockerfile.sh: ADD some missing files in SAGE_ROOT
​1b03fa9	src/pkgs/sage_conf-pypi/setup.py: Change SETENV so that it works in shells with set -e
​b3b54b7	Merge tag '9.3.rc2' into t/29039/pip_installable_package_sage_bootstrap
​ec10bc9	src/pkgs/sage_conf-pypi/setup.py: Fix for shell behavior on fedora
​278c36c	src/pkgs/sage_conf-pypi/setup.py: Use 'configure --enable-build-as-root' so it works in containers too
​cc8297c	Merge #29039

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​04da2c6	build/pkgs/ipywidgets: Patch out declaring install-requires of nbformat and jupyterlab-widgets
​94b8d3c	Merge #30913
​b54e77a	Merge #29039

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3f4a8ef	Trac #28890: Pass --disable-static to autoconf packages by default.
​8a8efee	Trac #28890: Don't install PARI's static library.
​cd39c76	Trac #28890: Don't use sdh_configure to configure zlib; it does not use
​9622d83	build/pkgs/zlib/spkg-install.in: Fixup
​3d701f8	build/pkgs/zlib/spkg-install.in: Handle errors from configure
​56178ab	build/bin/sage-dist-helpers (sdh_configure): On Cygwin, do not use --disable-static (until #30814 is done)
​4dcd135	Merge #28890

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c8ccab6	build/bin/sage-dist-helpers: Fixup: = instead of ==
​6f7819e	Merge #28890

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​df2f8d1	Merge tag '9.3' into t/29039/pip_installable_package_sage_bootstrap
​697e3c1	Merge #29039
​6ea676d	Merge tag '9.3' into t/31338/sage_setup__use_paths_within_sage_local_when_provided_via_sage_conf
​9ab6db9	Merge #31338

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​133aab7	Trac #29617: move MAXIMA_FAS to sage_conf.
​9b595c0	Trac #29617: modify Kenzo interface to support system ECL.
​b4ae35e	Trac #29617: treat empty MAXIMA_FAS and KENZO_FAS as None.
​2d40f36	Trac #29617: upgrade kenzo to v1.1.10.
​2078fa6	Merge tag '9.4.beta0' into t/31593/configure__paths_within__sage_local___prefix__for_sage_conf
​5cbe423	Merge #31593
​be48dd8	Merge #29617
​b4b3ec8	Merge tag '9.4.beta0' into t/29039/pip_installable_package_sage_bootstrap
​e1d2df0	Merge tag '9.4.beta1' into t/29039/pip_installable_package_sage_bootstrap
​ce51048	Merge #29039

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7e8a7a8	Makefile (sage-wheels): Use SETUPTOOLS_USE_DISTUTILS=local
​801e792	src/pkgs/sage_conf-relocatable/setup.py: Use wheel tags in build directory name too
​52d669d	src/pkgs/sage_conf-relocatable/setup.py: Do not use removed option --without-system-boost

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​55122f3

build/pkgs/pynac/spkg-install.in: Work around Trac #31920

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ff58be3

Makefile (sage-wheels): Fix wheel location in final message

[mkoeppe]

Installation with /usr/bin/python3 -m pip install -I --user https://github.com/sagemath/sage-wheels/releases/download/9.4.beta1/sage_conf-9.4b1-cp38-cp38-macosx_11_0_x86_64.whl https://github.com/sagemath/sage-wheels/releases/download/9.4.beta1/sagemath_standard-9.4b1-cp38-cp38-macosx_11_0_x86_64.whl fails with:

Collecting argon2-cffi (from notebook>=4.4.1->widgetsnbextension~=3.5.0->ipywidgets>=7.5.1->sagemath-standard==9.4b1)
  Downloading https://files.pythonhosted.org/packages/74/fd/d78e003a79c453e8454197092fce9d1c6099445b7e7da0b04eb4fe1dbab7/argon2-cffi-20.1.0.tar.gz (1.8MB)
[...]
Failed to build argon2-cffi

[dimpase]

#29039 is listed as a dependency, is it an error?

[mkoeppe]

No, that's correct

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

​76023cc	.gitignore: Ignore spkg/pkgs/*/{build,dist}
​8591334	src/pkgs/sage_conf-relocatable/setup.py: Move/copy/delete build trees more carefully
​adbbdc1	tox.ini (manylinux-python3...): Add EXTRA_SAGE_PACKAGES so that system python is accepted
​6a6a66a	Makefile (sage-wheels): Use SETUPTOOLS_USE_DISTUTILS=local
​64ff2e7	src/pkgs/sage_conf-relocatable/setup.py: Use wheel tags in build directory name too
​40050a2	src/pkgs/sage_conf-relocatable/setup.py: Do not use removed option --without-system-boost
​a2aa592	build/pkgs/pynac/spkg-install.in: Work around Trac #31920
​52ecb27	Makefile (sage-wheels): Fix wheel location in final message
​84637dd	Move sprc/pkgs/sage_conf-relocatable to pkgs/sage-conf_relocatable
​1c35e0f	Put pkgs/sage-conf in the root

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

​3b863d7	configure.ac: If --disable-notebook and --disable-sagelib, do not build zeromq etc.
​bfab056	tox.ini (local -- bash): Set PS1
​1b41f14	configure.ac: Add configure option --disable-doc
​e2522ff	tox.ini (docker), build/bin/write-dockerfile.sh: Handle EXTRA_SAGE_PACKAGES
​7507b22	build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no
​449fe19	build/pkgs/pynac/spkg-install.in: Explicitly use python3 in SAGE_VENV
​5d24f9d	build/bin/sage-spkg: Also source sage-src-env-config so that SAGE_VENV is available
​2dcbc90	src/pkgs/sage_conf-relocatable: New
​3bf0a49	Makefile: New targets pypi-sdists, sage-wheels
​59d5a9f	Move sprc/pkgs/sage_conf-relocatable to pkgs/sage-conf_relocatable

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b5267e8	configure.ac: Add configure option --disable-doc
​4da52e1	tox.ini (local -- bash): Set PS1
​f7bfefd	tox.ini (docker), build/bin/write-dockerfile.sh: Handle EXTRA_SAGE_PACKAGES
​92db7a5	build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no
​e914412	build/pkgs/pynac/spkg-install.in: Explicitly use python3 in SAGE_VENV
​a879f6d	build/bin/sage-spkg: Also source sage-src-env-config so that SAGE_VENV is available
​691512c	src/pkgs/sage_conf-relocatable: New
​99cdf97	Makefile: New target sage-wheels
​6b7bce6	Move sprc/pkgs/sage_conf-relocatable to pkgs/sage-conf_relocatable

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

​38a7040	configure.ac: If --disable-notebook and --disable-sagelib, do not build zeromq etc.
​f8fc4a7	configure.ac: Add configure option --disable-doc
​c830393	tox.ini (local -- bash): Set PS1
​aafb323	tox.ini (docker), build/bin/write-dockerfile.sh: Handle EXTRA_SAGE_PACKAGES
​c2ac2e8	build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no
​ac6fcde	build/pkgs/pynac/spkg-install.in: Explicitly use python3 in SAGE_VENV
​703c81a	build/bin/sage-spkg: Also source sage-src-env-config so that SAGE_VENV is available
​787d8d3	src/pkgs/sage_conf-relocatable: New
​a2861aa	Makefile: New target sage-wheels
​d9554e3	Move sprc/pkgs/sage_conf-relocatable to pkgs/sage-conf_relocatable

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​44b1e8d	Makefile (pypi-sdists): Add dependency on sage_setup
​98cded3	Merge branch 't/29039/pip_installable_package_sage_bootstrap' into t/31396/relocatable_wheel_version_of_package_sage_conf
​76f53c7	Makefile (sage-wheels), pkgs/sage-conf_relocatable/setup.py: Use 'pip wheel' instead of setup.py bdist_wheel; exclude all embedded trees; improve hints and output

[dimpase]

<deleted - meant to be on #31456>

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​569b188	Merge #29847
​35b8b74	Merge tag '9.4.beta5' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup
​773ef29	Merge #32073
​9aa1f10	Merge #29585
​8c5ee65	build/pkgs/pip/install-requires.txt: Update lower bound
​ed5f9f0	Merge tag '9.4.beta6' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install
​6b1b222	Merge #32046
​cfd0c25	Merge #29847
​40177f1	Merge #29039
​744b58e	Update for changed source layout

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​85d1d27	Merge tag '9.4.rc1' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install
​050f8df	build/pkgs/numpy/spkg-install.in: Use sdh_pip_install, sdh_prefix_args
​a41b507	Merge branch 't/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup
​e6e8ced	Merge #29847
​d69c496	build/pkgs/ptyprocess: pin version to 0.5.1
​7f13d04	Merge #32371
​bab6c0c	Merge #29039

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​290fa2f	build/pkgs/sagelib/src: Restore lost symlink
​478ffcd	Merge #32046
​b011389	Merge branch 't/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup' into t/29039/pip_installable_package_sage_bootstrap
​e4b1a95	Merge #29039

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​a23b1e1	pkgs/sage-conf_pypi/MANIFEST.in: Exclude build artifacts
​2f5be2d	configure.ac, build/pkgs/python3/spkg-configure.m4: New default --with-sage-venv=auto
​f700640	configure.ac: In config.status, create convenience symlinks prefix, venv
​b4a3845	m4/sage_spkg_collect.m4: If we install python3 spkg, install it in SAGE_VENV
​12cbbda	configure.ac: Keep old behavior of SAGE_VENV in incremental builds
​117fe3a	build/make/Makefile.in (...-clean): Do not fail trying to uninstall if SAGE_VENV has not been created yet
​ce1c344	Merge #32442
​e71b8a6	pkgs/sage-conf_pypi/setup.py: Key SAGE_ROOT only to Sage version and architecture
​9dcc189	pkgs/sage-conf_pypi/setup.py: Use --with-sage-venv
​e28f974	Merge #29039

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d626aa8	build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no
​82ce539	configure.ac: If --disable-notebook and --disable-sagelib, do not build zeromq etc.
​02505ca	pkgs/sage-conf_pypi/MANIFEST.in: Update from #31396
​aeca1a0	Makefile (pypi-sdists): Add sage_setup
​cd3dce9	pkgs/sage-conf_pypi/setup.py: Update directory of configured sage_conf.py
​c695921	pkgs/sage-conf/sage_conf.py.in (SAGE_SPKG_WHEELS): Update so it works if SAGE_VENV != SAGE_LOCAL
​1851bcb	Merge #29039

[dimpase]

On Linux, PATH in testing instructions should start from $HOME/.local/lib/python3.8 or something like this. There is no ~/Library, that's macOS-only.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​cebcda3	Merge tag '9.5.beta4' into t/29039/pip_installable_package_sage_bootstrap
​8e71c45	Merge branch 't/29039/pip_installable_package_sage_bootstrap' into t/31396/relocatable_wheel_version_of_package_sage_conf
​c524f6f	configure.ac (--disable-notebook): Also disable argon2_cffi
​5206f53	configure.ac (--disable-doc): New
​97b9fcf	configure.ac (--disable-doc): Disable also 'requests' and its dependencies, and direct sphinx dependencies
​18b4055	build/pkgs/pplpy/dependencies: Depend on sphinx only if SAGE_SPKG_INSTALL_DOCS!=no
​dfd23fb	build/pkgs/pplpy/dependencies: Fix up SAGE_SPKG_INSTALL_DOCS logic
​7307c75	configure.ac (--disabled-notebook): Also disable nbclient
​2901e33	Merge #32759

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

​687e5ba	src/sage/features/sagemath.py: Add doctests
​40a9760	build/make/Makefile.in: Raise an error if installation into disabled installation tree is attempted
​51f3002	build/make/Makefile.in: Better error message when docbuild is attempted but configure --disable-doc has been used
​b9da161	Merge tag '9.5.beta6' into t/32759/__configure___disable_doc
​0026892	Merge tag '9.5.beta5' into t/29039/pip_installable_package_sage_bootstrap
​b5cf9f6	Merge #29039
​04df2b7	build/make/Makefile.in (all-start): Use all-sage-docs, not doc
​61d97f7	Merge #32759
​1a36c77	tox.ini (macos-nohomebrew): Take cmake from homebrew, remove outdated configure options, remove Python3.framework/Headers from CPATH
​15902ee	pkgs/sage-conf_relocatable/setup.py: Update configure args

[mkoeppe]

Stalled in needs_review or needs_info; likely won't make it into Sage 9.5.

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​0df932d	src/sage/misc/sageinspect.py: Make doctest pass even if sphinx formatting is not available
​6f8f8e0	src/sage/repl/ipython_tests.py: Make doctest pass even if sphinx formatting is not available
​1807446	src/sage/misc/package.py: Use gmpy2 instead of alabaster as example in doctest
​f4aaa81	README.md, build/make/install: Update - documentation no longer has special log file location
​651c986	build/make/Makefile.in (doc): Revert to just doc-html (not both doc-html and doc-pdf)
​edaa3e6	Merge tag '9.5.beta8' into t/32759/__configure___disable_doc
​a7fa5aa	Merge #29039
​2dd7cf5	Merge tag '9.5.beta9' into t/32759/__configure___disable_doc
​2e0b91d	src/sage/doctest/control.py: Fix up merge
​b572416	Merge #32759

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​049a5f6	tox.ini: Do not set environment variable CONDARC
​f3116a1	tox.ini (conda): Force use of conda's python3
​6418579	Merge #32113
​933134c	build/pkgs/gfortran/distros/homebrew.txt: Use gfortran
​1aa9328	Merge #32947
​e99a09e	sed -i.bak 's/ubunty/ubuntu/g' .github/workflows/*.yml
​edb4364	.github/workflows/tox.yml: Replace homebrew-macos-python3_xcode-standard by homebrew-macos-usrlocal-python3_xcode-standard
​e27e161	Merge #32703
​96a45d1	tox.ini, .github/workflows: Remove ubuntu-groovy
​f3bf7ea	Merge #33068

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ed73fe3

configure.ac: Clean up duplication of --disable-... options

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6ce82f7

Makefile (sage-wheels): Remove obsolete --use-feature=in-tree-build

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​14e83dc

tox.ini: Add macos-11.3, macos-12.1

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bddc117

pkgs/sage-conf_relocatable/sage_conf.py.in: Update from pkgs/sage-conf/sage_conf.py.in

[mkoeppe]

To avoid the duplication between the two versions of sage_conf.py.in, could do what was suggested in ​https://trac.sagemath.org/ticket/29039#comment:148 - split module sage_conf into several modules of a sage_conf package

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​83c749e

Split pkgs/sage-conf/sage_conf.py.in

[mkoeppe]

Need to get uninstallation of script package sage_conf fixed, or switching between branches with sage_conf.py and branches with sage_conf/ will be tricky

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0f998d2

Merge tag '9.5.rc0' into t/31396/relocatable_wheel_version_of_package_sage_conf

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e913510

pkgs/sage-conf*/: Use sage_conf.py and _sage_conf/

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​8da2c8a

pkgs/sage-conf*/: Use sage_conf.py and _sage_conf/

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​4079962

pkgs/sage-conf*/: Use sage_conf.py and _sage_conf/

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​06bbe15

pkgs/sage-conf*/: Use sage_conf.py and _sage_conf/

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​755338a

pkgs/sage-conf*/: Use sage_conf.py and _sage_conf/

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​e142d32

pkgs/sage-conf*/: Use sage_conf.py and _sage_conf/

[mkoeppe]

tox -e local-macos-10.15-nohomebrew-python3_xcode -- SAGE_CHECK=no sage-wheels currently builds a mistagged sage_conf-9.5rc0-cp38-cp38-macosx_11_0_x86_64.whl

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​61aa731

Merge tag '9.5.rc2' into t/31396/relocatable_wheel_version_of_package_sage_conf

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3b61796

pkgs/sage-conf_relocatable/setup.py: Remove --with-mp=gmp

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9e2d6fc

pkgs/sage-conf_relocatable/sage_conf.py: Make sage_root package_data of _sage_conf

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2b9cf2a

pkgs/sage-conf_pypi/sage_conf.py: Fix up SAGE_ROOT_ABS

[mkoeppe]

Replying to mkoeppe:

tox -e local-macos-10.15-nohomebrew-python3_xcode -- SAGE_CHECK=no sage-wheels currently builds a mistagged sage_conf-9.5rc0-cp38-cp38-macosx_11_0_x86_64.whl

That was caused by trash in pkgs/sage-conf_relocatable/build/

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​903408b

pkgs/sage-conf_relocatable/setup.py: Remove debugging output

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​43fdf0d

Merge tag '9.5' into t/31396/relocatable_wheel_version_of_package_sage_conf

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​51191fe	.github/workflows/tox.yml (local-macos): Fix filtering of experimental packages
​12f88cf	.github/workflows/tox.yml (local-macos): Group optional/experimental package builds into fewer jobs
​49236dd	tox.ini: Add local-macos-python3_pythonorg-python3.9 etc., refactor CONFIG_CONFIGURE_ARGS_1=...--with-python
​1fcdc91	tox.ini: Updates for macos and manylinux from #31396
​41ec3aa	tox.ini: Reindent
​0e1323c	Merge #33140

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​70c2e95	pkgs/sage-conf*: Refactor
​ffb5344	Merge tag '9.6.rc3' into t/33295/refactor_sage_conf
​aa75075	Merge #33295

[saraedum]

There seem to be merge conflicts.

[saraedum]

Do you want this to be reviewed & merged even though it does not work on Linux yet? Or does it work by now?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fbb318d	Merge tag '9.7.beta2' into t/33295/refactor_sage_conf
​7bed79d	Merge #33295

[mkoeppe]

Replying to saraedum:

Do you want this to be reviewed & merged even though it does not work on Linux yet?

Yes, that would be great!

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​93ca1f7	Merge tag '9.7.beta3' into t/33295/refactor_sage_conf
​9d10055	pkgs/sage-conf*: Remove some obsolete files
​eeddb03	pkgs/sage-conf/setup.cfg: Fix up console_scripts
​8297915	Merge #33295