Use pip --use-feature=in-tree-build, replace use of sdh_setup_bdist_wheel by sdh_pip_install

[mkoeppe]

This is a feature added in pip 21.1.

We can now build wheels for all packages using pip wheel, even those with symlinks or that need in-tree configuration files. Direct invocation of setup.py bdist_wheel is no longer necessary.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a1e78f2	build/pkgs/pillow/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install, fix quoting of CONDA_PREFIX with whitespace
​6166a8c	build/bin/sage-dist-helpers: Generalize sdh_globaloptionify to sdh_prefix_args
​2dae827	build/pkgs/jupyter_jsmol/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install
​f358345	build/pkgs/python_igraph/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a1d33bc	build/pkgs/pillow/spkg-install.in, build/bin/sage-dist-helpers: Remove debugging code
​413f6b5	build/pkgs/{jupyter_jsmol,python_igraph,pillow}/spkg-install.in: Use --build-option, not --global-option

[mkoeppe]

A similar change can be done with numpy, but to avoid conflicts, that's best done in #32021 or #31565

[jhpalmieri]

Do we need to change build/pkgs/pip/install-requires.txt to require at least version 21.1?

[mkoeppe]

Yes, it's a good idea to update it to document this lower bound - although technically pip's lower bound is not used - pip does not appear in build/pkgs/sagelib/src/pyproject.toml.m4 nor build/pkgs/sagelib/src/setup.cfg.m4.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8c5ee65

build/pkgs/pip/install-requires.txt: Update lower bound

[jhpalmieri]

Replying to mkoeppe:

Yes, it's a good idea to update it to document this lower bound - although technically pip's lower bound is not used - pip does not appear in build/pkgs/sagelib/src/pyproject.toml.m4 nor build/pkgs/sagelib/src/setup.cfg.m4.

So what happens if someone has a system version of pip installed that doesn't support --use-feature=in-tree-build?

[mkoeppe]

We don't use any system Python packages, even when we use system Python.

[jhpalmieri]

Are you saying that we always build and use our own pip? Are the entries in pip/distros ignored?

[mkoeppe]

Replying to jhpalmieri:

Are you saying that we always build and use our own pip?

That's right. See build/bin/sage-venv, which is the script that we use to set up our venv over the system Python. ​https://docs.python.org/3/library/venv.html#venv.EnvBuilder with_pip defaults to False; and we also use system_site_packages=False. We then bootstrap setuptools, pip, and wheel in our empty virtual environment -- same as we do when we build our own python.

Are the entries in pip/distros ignored?

Yes, they are only decoration. There is currently only one use-case for Python system packages, and that is a conda development environment (see src/environment.yml).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ed5f9f0

Merge tag '9.4.beta6' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​85d1d27

Merge tag '9.4.rc1' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​050f8df

build/pkgs/numpy/spkg-install.in: Use sdh_pip_install, sdh_prefix_args

[mkoeppe]

Replying to mkoeppe:

A similar change can be done with numpy, but to avoid conflicts, that's best done in #32021 or #31565

I've made the change now

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​290fa2f

build/pkgs/sagelib/src: Restore lost symlink

[jhpalmieri]

Okay, let's merge it. It works for me on OS X. I haven't tested with conda, but the changes make sense.

[mkoeppe]

Thanks!