#32046 closed enhancement (fixed)

Use pip --use-feature=in-tree-build, replace use of sdh_setup_bdist_wheel by sdh_pip_install

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.5
Component: build Keywords:
Cc: jhpalmieri, gh-kliem, isuruf Merged in:
Authors: Matthias Koeppe Reviewers: John Palmieri
Report Upstream: N/A Work issues:
Branch: 290fa2f (Commits, GitHub, GitLab) Commit: 290fa2ff50ca053d726bb0a1a8e0470a66040c26
Dependencies: Stopgaps:

Status badges

Description

This is a feature added in pip 21.1.

We can now build wheels for all packages using pip wheel, even those with symlinks or that need in-tree configuration files. Direct invocation of setup.py bdist_wheel is no longer necessary.

Change History (22)

comment:1 Changed 14 months ago by mkoeppe

  • Branch set to u/mkoeppe/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

comment:2 Changed 14 months ago by git

  • Commit set to f3583457677284130daed94f8a96e480714fcec9

Branch pushed to git repo; I updated commit sha1. New commits:

a1e78f2build/pkgs/pillow/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install, fix quoting of CONDA_PREFIX with whitespace
6166a8cbuild/bin/sage-dist-helpers: Generalize sdh_globaloptionify to sdh_prefix_args
2dae827build/pkgs/jupyter_jsmol/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install
f358345build/pkgs/python_igraph/spkg-install.in: Switch from sdh_setup_bdist_wheel to sdh_pip_install

comment:3 Changed 14 months ago by git

  • Commit changed from f3583457677284130daed94f8a96e480714fcec9 to 413f6b5c4be4286f6b7c411e498129b16fbc77f0

Branch pushed to git repo; I updated commit sha1. New commits:

a1d33bcbuild/pkgs/pillow/spkg-install.in, build/bin/sage-dist-helpers: Remove debugging code
413f6b5build/pkgs/{jupyter_jsmol,python_igraph,pillow}/spkg-install.in: Use --build-option, not --global-option

comment:4 follow-up: Changed 14 months ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Cc isuruf added
  • Status changed from new to needs_review

A similar change can be done with numpy, but to avoid conflicts, that's best done in #32021 or #31565

comment:5 Changed 14 months ago by jhpalmieri

Do we need to change build/pkgs/pip/install-requires.txt to require at least version 21.1?

comment:6 follow-up: Changed 14 months ago by mkoeppe

Yes, it's a good idea to update it to document this lower bound - although technically pip's lower bound is not used - pip does not appear in build/pkgs/sagelib/src/pyproject.toml.m4 nor build/pkgs/sagelib/src/setup.cfg.m4.

comment:7 Changed 14 months ago by git

  • Commit changed from 413f6b5c4be4286f6b7c411e498129b16fbc77f0 to 8c5ee65a8872549cdfcce321bbee8bef81b57ccb

Branch pushed to git repo; I updated commit sha1. New commits:

8c5ee65build/pkgs/pip/install-requires.txt: Update lower bound

comment:8 in reply to: ↑ 6 Changed 14 months ago by jhpalmieri

Replying to mkoeppe:

Yes, it's a good idea to update it to document this lower bound - although technically pip's lower bound is not used - pip does not appear in build/pkgs/sagelib/src/pyproject.toml.m4 nor build/pkgs/sagelib/src/setup.cfg.m4.

So what happens if someone has a system version of pip installed that doesn't support --use-feature=in-tree-build?

comment:9 Changed 14 months ago by mkoeppe

We don't use any system Python packages, even when we use system Python.

comment:10 follow-up: Changed 14 months ago by jhpalmieri

Are you saying that we always build and use our own pip? Are the entries in pip/distros ignored?

comment:11 in reply to: ↑ 10 Changed 14 months ago by mkoeppe

Replying to jhpalmieri:

Are you saying that we always build and use our own pip?

That's right. See build/bin/sage-venv, which is the script that we use to set up our venv over the system Python. https://docs.python.org/3/library/venv.html#venv.EnvBuilder with_pip defaults to False; and we also use system_site_packages=False. We then bootstrap setuptools, pip, and wheel in our empty virtual environment -- same as we do when we build our own python.

Are the entries in pip/distros ignored?

Yes, they are only decoration. There is currently only one use-case for Python system packages, and that is a conda development environment (see src/environment.yml).

comment:12 Changed 13 months ago by mkoeppe

  • Status changed from needs_review to needs_work
  • Work issues set to rebase

comment:13 Changed 13 months ago by git

  • Commit changed from 8c5ee65a8872549cdfcce321bbee8bef81b57ccb to ed5f9f0c9219a86a85cd46df588414c920b063b9

Branch pushed to git repo; I updated commit sha1. New commits:

ed5f9f0Merge tag '9.4.beta6' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

comment:14 Changed 13 months ago by mkoeppe

  • Milestone changed from sage-9.4 to sage-9.5
  • Status changed from needs_work to needs_review

comment:15 Changed 13 months ago by mkoeppe

  • Work issues rebase deleted

comment:16 Changed 12 months ago by git

  • Commit changed from ed5f9f0c9219a86a85cd46df588414c920b063b9 to 85d1d2716188e67b85b48838c52d8f919f7244ef

Branch pushed to git repo; I updated commit sha1. New commits:

85d1d27Merge tag '9.4.rc1' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

comment:17 Changed 12 months ago by git

  • Commit changed from 85d1d2716188e67b85b48838c52d8f919f7244ef to 050f8df9a056a966e4fed9371ddb6244d0cb18f2

Branch pushed to git repo; I updated commit sha1. New commits:

050f8dfbuild/pkgs/numpy/spkg-install.in: Use sdh_pip_install, sdh_prefix_args

comment:18 in reply to: ↑ 4 Changed 12 months ago by mkoeppe

Replying to mkoeppe:

A similar change can be done with numpy, but to avoid conflicts, that's best done in #32021 or #31565

I've made the change now

comment:19 Changed 12 months ago by git

  • Commit changed from 050f8df9a056a966e4fed9371ddb6244d0cb18f2 to 290fa2ff50ca053d726bb0a1a8e0470a66040c26

Branch pushed to git repo; I updated commit sha1. New commits:

290fa2fbuild/pkgs/sagelib/src: Restore lost symlink

comment:20 Changed 12 months ago by jhpalmieri

  • Reviewers set to John Palmieri
  • Status changed from needs_review to positive_review

Okay, let's merge it. It works for me on OS X. I haven't tested with conda, but the changes make sense.

comment:21 Changed 12 months ago by mkoeppe

Thanks!

comment:22 Changed 12 months ago by vbraun

  • Branch changed from u/mkoeppe/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install to 290fa2ff50ca053d726bb0a1a8e0470a66040c26
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.