Opened 23 months ago
Closed 14 months ago
#31535 closed enhancement (fixed)
New make targets "sagelib-tox-...", "sage_docbuild-tox-...", ...
Reported by: | mkoeppe | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.5 |
Component: | porting | Keywords: | |
Cc: | jhpalmieri, dimpase, gh-kliem, mjo, gh-tobiasdiez | Merged in: | |
Authors: | Matthias Koeppe | Reviewers: | Michael Orlitzky |
Report Upstream: | N/A | Work issues: | |
Branch: | ad12d25 (Commits, GitHub, GitLab) | Commit: | ad12d251cc7a6a5a7a5a369310bc1e7ad89b8f66 |
Dependencies: | #32899 | Stopgaps: |
Description (last modified by )
For all script packages SPKG
that have an embedded source tree -- currently sagelib
, sage_docbuild
, sage_sws2rst
-- we add a new target:
SPKG-tox-%
(which callstox -e %
in the package's source tree)
These targets are particularly useful for invoking them within the portability tests of the top-level tox
:
$ TARGETS_PRE=config.status tox -e docker-ubuntu-focal-standard -- sage_docbuild-tox
invokes the tox testing of thesage_docbuild
distribution$ EXTRA_CONFIGURE_ARGS="--disable-notebook" tox -e docker-ubuntu-focal-standard -- build sagelib-tox-python-sagewheels-nopypi
builds (in a Docker container) the Sage distribution (without Jupyter notebook) and then invokes the tox testing of sagelib (added in #30913) in a venv populated using the wheels$ tox -e local-homebrew-macos-maximal -- build-local cysignals networkx sagelib-tox-python-sagewheels-sitepackages-norequirements-nopypi
(added in #30913) builds the non-Python parts of the Sage distribution on top of Homebrew, using a virtual environment that uses homebrew's Python packages$ tox -e local-conda-forge-macos-src-environment -- sage_conf sagelib-tox-python-sagewheels-sitepackages-norequirements-nobuildisolation-nopypi
(needs #30913, #31384) uses thesrc/environment.yml
file to set up a complete environment from conda and then tests sagelib in a venv using tox
Change History (27)
comment:1 Changed 23 months ago by
Branch: | → u/mkoeppe/new_make_targets__sagelib_sdist____sage_docbuild_sdist_______and__sagelib_tox__________ |
---|
comment:2 Changed 23 months ago by
Authors: | → Matthias Koeppe |
---|---|
Cc: | jhpalmieri dimpase gh-kliem added |
Commit: | → 1eef6d6a8a2eeff9508b936ac8ececbf9f60f362 |
Status: | new → needs_review |
comment:3 Changed 23 months ago by
Commit: | 1eef6d6a8a2eeff9508b936ac8ececbf9f60f362 → 69649337a75524486c51aff2547f0b8ead9e36cf |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
6964933 | build/make/Makefile.in: fixup
|
comment:4 Changed 23 months ago by
Description: | modified (diff) |
---|
comment:5 Changed 23 months ago by
Description: | modified (diff) |
---|
comment:6 Changed 23 months ago by
Description: | modified (diff) |
---|
comment:7 Changed 23 months ago by
Commit: | 69649337a75524486c51aff2547f0b8ead9e36cf → 188bf2e3bd076e5b1966422c10e7b9dae5affd36 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
9993348 | build/make/Makefile.in: Run targets SPKG-tox, SPKG-tox-% outside of sage-env, using SAGE_ORIG_PATH - so that tox from conda works
|
4533018 | tox.ini: Add local-conda-environment-src
|
188bf2e | tox.ini, src/doc/bootstrap: Get tox into standard builds
|
comment:8 Changed 23 months ago by
Description: | modified (diff) |
---|
comment:9 Changed 23 months ago by
Description: | modified (diff) |
---|
comment:10 Changed 23 months ago by
Status: | needs_review → needs_work |
---|
comment:11 Changed 22 months ago by
Milestone: | sage-9.3 → sage-9.4 |
---|
Moving this ticket to 9.4, as it seems unlikely that it will be merged in 9.3, which is in the release candidate stage
comment:12 Changed 19 months ago by
Milestone: | sage-9.4 → sage-9.5 |
---|
Setting a new milestone for this ticket based on a cursory review.
comment:13 Changed 16 months ago by
Dependencies: | → #32442, #32698 |
---|
comment:14 Changed 16 months ago by
Commit: | 188bf2e3bd076e5b1966422c10e7b9dae5affd36 → bca214134dc225603801470f7bb2999e9e6bdbca |
---|
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
8af6041 | Merge tag '9.5.beta2' into t/32442/__configure__make___with_sage_venv_the_default
|
b3f7d34 | .gitignore: Add /venv
|
fc4b641 | configure.ac: Remove conveniene symlinks before (re)creating them
|
8bcbf1d | Makefile (distclean): Remove convenience symlinks prefix, venv
|
dcb4a08 | configure.ac: Indicate default in help string for --with-sage-venv
|
e33bc89 | Merge tag '9.5.beta3' into t/32442/__configure__make___with_sage_venv_the_default
|
2101b8c | build/pkgs/python3/spkg-build.in: Make sure that python finds sqlite3 when determining which extension modules to build
|
3bbc5d8 | build/pkgs/python3/spkg-build.in: Set rpath
|
124b605 | Merge #32698
|
bca2141 | pkgs/sagemath-standard/tox.ini: Use SAGE_VENV or venv symlink to find wheels
|
comment:15 Changed 16 months ago by
Cc: | mjo gh-tobiasdiez added |
---|---|
Status: | needs_work → needs_review |
comment:16 Changed 16 months ago by
Commit: | bca214134dc225603801470f7bb2999e9e6bdbca → bdbab3aca65138c1a80009c546df53af04d740f6 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
bdbab3a | build/make/Makefile.in: Remove SPKG-tox, SPKG-sdist targets
|
comment:17 Changed 16 months ago by
Description: | modified (diff) |
---|---|
Summary: | New make targets "sagelib-sdist", "sage_docbuild-sdist", ... and "sagelib-tox-...", ... → New make targets "sagelib-tox-...", "sage_docbuild-tox-...", ... |
Reduced the scope of the ticket
comment:18 Changed 14 months ago by
I've tried to test these locally but I always get some complaint about sage_setup.py
. For example,
$ EXTRA_CONFIGURE_ARGS="--without-system-python3" tox -e local-direct -- build sagelib-tox-python-sagewheels
This proceeded to build everything inside of .tox/local-direct
, but then...
cd '/home/mjo/src/sage.git/build/pkgs/sagelib/src' && export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/12/bin:/home/mjo/bin:/home/mjo/bin" && SAGE_SPKG_WHEELS=/home/mjo/src/sage.git/.tox/local-direct/local/var/lib/sage/wheels tox -v -v -v -e python-sagewheels using tox.ini: /home/mjo/src/sage.git/pkgs/sagemath-standard/tox.ini (pid 12133) removing /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/log using tox-3.24.4 from /usr/lib/python3.9/site-packages/tox/__init__.py (pid 12133) GLOB start: packaging GLOB sdist-make: /home/mjo/src/sage.git/pkgs/sagemath-standard/setup.py [12140] /home/mjo/src/sage.git/pkgs/sagemath-standard$ /usr/bin/python3.9 setup.py sdist --formats=zip --dist-dir /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/dist >.tox/log/GLOB-0.log ERROR: invocation failed (exit code 1), logfile: /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/log/GLOB-0.log ================================== log start =================================== Traceback (most recent call last): File "/home/mjo/src/sage.git/pkgs/sagemath-standard/setup.py", line 31, in <module> from sage_setup.excepthook import excepthook ModuleNotFoundError: No module named 'sage_setup' =================================== log end ==================================== ERROR: FAIL could not package project - v = InvocationError('/usr/bin/python3.9 setup.py sdist --formats=zip --dist-dir /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/dist', 1) make[1]: *** [Makefile:2798: sagelib-tox-python-sagewheels] Error 2 make[1]: Leaving directory '/home/mjo/src/sage.git/build/make' real 0m4.988s user 0m1.223s sys 0m0.145s *************************************************************** Error building Sage. The following package(s) may have failed to build (not necessarily during this run of 'make sagelib-tox-python-sagewheels'): It is safe to delete any log files and build directories, but they contain information that is helpful for debugging build problems. WARNING: If you now run 'make' again, the build directory of the same version of the package will, by default, be deleted. Set the environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this. make: *** [Makefile:39: sagelib-tox-python-sagewheels] Error 1 ERROR: InvocationError for command /bin/bash -c 'export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/bin:/usr/lib/llvm/12/bin:/home/mjo/bin:/home/mjo/bin && : && case "" in 1|y*|Y*);; *) ./bootstrap ;; esac && : && case "" in 1|y*|Y*);; *) ./configure --prefix=/home/mjo/src/sage.git/.tox/local-direct/local --enable-experimental-packages --enable-download-from-upstream-url --with-system-python3=yes --without-system-python3 ;; esac && case "build sagelib-tox-python-sagewheels" in bash) bash -i; exit ;; config*) ;; *) make -k V=0 base-toolchain ;; esac && make -k V=0 SAGE_CHECK=warn SAGE_CHECK_PACKAGES="!cython,!r,!python3,!gap,!cysignals,!linbox,!git,!ppl,!cmake,!rpy2,!sage_sws2rst" build sagelib-tox-python-sagewheels && ( [ -z "" ] || make -k V=0 SAGE_CHECK=warn SAGE_CHECK_PACKAGES="!cython,!r,!python3,!gap,!cysignals,!linbox,!git,!ppl,!cmake,!rpy2,!sage_sws2rst" || echo "(error ignored)" ) ' (exited with code 2) __________________________________________________________________________________ summary __________________________________________________________________________________ ERROR: local-direct: commands failed
Is that expected (did I construct my example wrong?)
comment:19 Changed 14 months ago by
Dependencies: | #32442, #32698 → #32899 |
---|
comment:20 Changed 14 months ago by
Commit: | bdbab3aca65138c1a80009c546df53af04d740f6 → ad12d251cc7a6a5a7a5a369310bc1e7ad89b8f66 |
---|
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
e5ad518 | pkgs/sagemath-standard/setup.py: Do not depend on sage_setup for sdist or egg_info
|
821d16e | src/doc/en/developer/packaging_sage_library.rst: Add bootstrap to testing instructions to avoid the trap of #32868
|
77c957f | src/doc/en/developer/packaging_sage_library.rst: Put hierarchy section one level higher
|
9375ef8 | pkgs/sagemath-standard/tox.ini: Use SAGE_VENV or venv symlink to find wheels
|
c6fc111 | Prettier diagram
|
7d6a44c | Use :mod: as markup for packages/modules
|
f83d424 | Improve ABC example
|
817a8d0 | src/doc/en/developer/packaging_sage_library.rst: More :mod: and :class: markup
|
b555735 | src/doc/en/developer/packaging_sage_library.rst: Link to pypi.org and to documentation of packaging metadata
|
ad12d25 | Merge #32899
|
comment:23 Changed 14 months ago by
I just noticed these as well, probably from an empty $CC --version
in sage_setup. No idea if they're new, I probably just happened to be looking in the right place at exactly the right time:
Getting requirements to build wheel: finished with status 'done' Preparing wheel metadata: started Running command /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/python-s\ agewheels-nopypi/bin/python /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/\ python-sagewheels-nopypi/lib/python3.9/site-packages/pip/_vendor/pep517/in_proc\ ess/_in_process.py prepare_metadata_for_build_wheel /tmp/tmphl6_9kin /bin/sh: 1: --version: not found
Building wheel for sagemath-standard (PEP 517): started Running command /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/python-sag\ ewheels-nopypi/bin/python /home/mjo/src/sage.git/pkgs/sagemath-standard/.tox/py\ thon-sagewheels-nopypi/lib/python3.9/site-packages/pip/_vendor/pep517/in_proces\ s/_in_process.py build_wheel /tmp/tmpqovldekw /bin/sh: 1: --version: not found
comment:25 Changed 14 months ago by
Reviewers: | → Michael Orlitzky |
---|---|
Status: | needs_review → positive_review |
Looking better now; regardless, the Makefile
targets themselves are working as intended.
comment:27 Changed 14 months ago by
Branch: | u/mkoeppe/new_make_targets__sagelib_sdist____sage_docbuild_sdist_______and__sagelib_tox__________ → ad12d251cc7a6a5a7a5a369310bc1e7ad89b8f66 |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
New commits:
build/make/Makefile.in: Add targets SPKG-sdist, SPKG-tox, SPKG-tox-% for script packages