tox.ini: Add local-macos-nohomebrew environments, deployment targets 10.14, 10.15, 11.1

[mkoeppe]

This will do a "best effort" isolation to avoid using a homebrew installation in /usr/local for anything except bootstrapping and unpacking xz tarballs + a little trick for liblzma.h, which is missing on macOS.

To test:

$ EXTRA_CONFIGURE_ARGS="--disable-r" tox -e local-macos-nohomebrew-python3_xcode

It can then be checked that no libraries from /usr/local leak in, using

find prefix/lib -name "*.so" -o -name "*.dylib*" -exec otool -L {} \;

(Building R is disabled because the R package build still pokes around in /usr/local and finds libintl and liblzma.)

To set specific deployment targets, use:

$ EXTRA_CONFIGURE_ARGS="--disable-r" tox -e local-macos-10.14-nohomebrew-python3_xcode -- config.status
$ EXTRA_CONFIGURE_ARGS="--disable-r" tox -e local-macos-10.15-nohomebrew-python3_xcode -- config.status
$ EXTRA_CONFIGURE_ARGS="--disable-r" tox -e local-macos-11.1-nohomebrew-python3_xcode -- config.status

... and with python3.7 from XCode:

$ tox -e local-macos-10.14-nohomebrew-python3_xcode-python3.7 -- config.status

(Remove config.status from the end of these lines to have the actual build done.)

These environments allow building Sage for older macOS versions, for example for building a 10.14 distribution on a 10.15 machine. The wheels are then correctly tagged, for example pplpy-0.8.6-cp38-cp38-macosx_10_14_x86_64.whl.

Cannot go in the opposite direction (building for 11 on a 10.15 machine) -- because pip will refuse to install a previously built wheel that is tagged for 11.)

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​87e2dba

tox.ini: Add local-macos-nohomebrew and configuration facors macos-{10.14,10.15,11.1}

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1ed9302

tox.ini: Add macos-{python3_xcode,nohomebrew}-python{3.7,3.8}

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​0458f23

tox.ini: Add macos-{python3_xcode,nohomebrew}-python{3.7,3.8}

[mkoeppe]

This is best tested together with #31552, #31562, which do some of the necessary isolation work.

More isolation work is needed at least for pillow and freetype - as observed in ​https://trac.sagemath.org/ticket/31396#comment:32

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8b5b273	build/pkgs/singular/checksums.ini: Use 4.2.0p1+2021-03-24+sage-2
​39629dd	Merge #31552
​1ef46d0	build/pkgs/giac: Add another patch in patches/autotools
​89f61d2	build/pkgs/giac: Update to 1.6.0.47p3
​9d9c562	Merge #31562

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​236ea3c

tox.ini (macos-nohomebrew): Pass compiler configuration to configure using variables other than CC, CXX.

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​92db4e4

tox.ini (macos-nohomebrew): Pass compiler configuration to configure using variables other than CC, CXX.

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​737b21c	src/sage/stats/r.py: Mark all 2 doctests in this file as # optional - rpy2
​4403924	src/sage/interfaces/r.py: Mark all tests # optional - rpy2
​405ebb9	More # optional - rpy2
​b7c1576	Merge tag '9.3.beta8' into t/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional
​ce3b35f	More # optional - r
​49c10c1	Revert "build/pkgs/{r,rpy2}: Downgrade to optional"
​a67300b	Merge #30383
​f0a5fb5	configure.ac: Add option --disable-r
​f04c134	Merge branch 't/30383/new_package_type__optional_enabled_by_default' into t/31409/cygwin_standard__r_build_fails_____downgrade_r__rpy2_to_optional
​7966b66	Merge #31409

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b5edde2

tox.ini (macos-nohomebrew): Fix typo in PILLOW_...

[mkoeppe]

Now pillow does not find zlib

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​dffcdbc

tox.ini (macos-nohomebrew): Fix up pillow build by setting ZLIB_ROOT

[mkoeppe]

pillow still finds various libraries in /usr/local because of very creative discovery code that finds everything in /usr/local because we told it that it can have /usr/local/opt/xz/include - see _add_directory (​https://github.com/python-pillow/Pillow/blob/8.1.x/setup.py#L197)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3779a25

tox.ini (macos-nohomebrew): Disable more libraries to remove dependencies on homebrew

[mkoeppe]

Fixed pillow. Now I can build a wheel in #31396 that does not have any dependencies on shared libraries in /usr/local.

[jhpalmieri]

For what it's worth, I see a few suspicious items in the log files. (This is on a system with homebrew and with source .homebrew-build-env in my shell init file. That may be causing some of these.)

• cvxopt includes flags -L/usr/local/lib and -I/usr/local/include when calling gcc
• pillow says Appending path /usr/local/Cellar/xz/5.2.5/include and then the gcc flag -I/usr/local/Cellar/xz/5.2.5/include.

[mkoeppe]

The xz include is on purpose -- see comment added in tox.ini

[mkoeppe]

Replying to jhpalmieri:

For what it's worth, I see a few suspicious items in the log files. (This is on a system with homebrew and with source .homebrew-build-env in my shell init file. That may be causing some of these.)

Yes, this a good setting for testing this ticket. The build using tox is designed to be isolated from the effects of .homebrew-build-env.

• cvxopt includes flags -L/usr/local/lib and -I/usr/local/include when calling gcc

This is probably ​https://github.com/cvxopt/cvxopt/blob/master/setup.py#L55 in combination with a bug in our build system - #31584.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7ff26df	git grep -l SAGE_SUITESPARSE | xargs sed -i .bak s/SAGE_SUITESPARSE_LOCALINSTALL/SAGE_SUITESPARSE_PREFIX/g
​52537d1	build/pkgs/cvxopt/spkg-install.in: Actually use the value of SAGE_SUITESPARSE_PREFIX
​ed1a089	Merge #31584

[jhpalmieri]

Okay, this looks good to me.

[mkoeppe]

Thanks!

[jhpalmieri]

By the way, should commands like make distclean also clean the .tox directory?

[mkoeppe]

I think it should not because .tox is not a "build artifact" - if this reasoning makes sense.

[jhpalmieri]

That makes sense. Maybe I need to learn more about tox. What command should I use if I want to restart EXTRA_CONFIGURE_ARGS="--disable-r" tox -e local-macos-nohomebrew-python3_xcode from scratch? Just delete .tox/local-macos-nohomebrew-python3?

[mkoeppe]

Replying to jhpalmieri:

That makes sense. Maybe I need to learn more about tox. What command should I use if I want to restart EXTRA_CONFIGURE_ARGS="--disable-r" tox -e local-macos-nohomebrew-python3_xcode from scratch? Just delete .tox/local-macos-nohomebrew-python3?

That works, but you can also pass the -r option to tox, which will start from scratch.

[jhpalmieri]

Great, thanks!