Opened 19 months ago
Last modified 11 days ago
#29039 needs_review enhancement
pipinstallable version of package sage_conf  installs nonPython bits of the Sage distribution in ~/.sage/
Reported by:  mkoeppe  Owned by:  

Priority:  critical  Milestone:  sage9.5 
Component:  build  Keywords:  sd111 
Cc:  jhpalmieri, dimpase, fbissey  Merged in:  
Authors:  Matthias Koeppe  Reviewers:  Dima Pasechnik 
Report Upstream:  N/A  Work issues:  
Branch:  u/mkoeppe/pip_installable_package_sage_bootstrap (Commits, GitHub, GitLab)  Commit:  cfd0c254d78f2f5fb5004e168235e5b520e43717 
Dependencies:  #31577, #32057, #29847, #32071  Stopgaps: 
Description (last modified by )
As #30913 makes sage_conf
both (a) a PEP517/518 build_system
requires
and (b) an installrequires
of sagelib (sagemathstandard
), also a version of sage_conf
must be made available on PyPI.
This version of sage_conf
includes a copy of SAGE_ROOT
and SAGE_ROOT/build/**
in its source distribution. On installation (or wheel building) it establishes a pythonless SAGE_LOCAL hierarchy (#31362) in ~/.sage/
by running configure
and make buildlocal
; and then installs the configured sage_conf.py
(#29038) as a Python module (and sageconfig
, sageenvconfig
).
There are no changes to how the Sage distribution is built. This ticket only provides an alternative and strictly optional way of installing (the nonPython bits of) Sagethedistribution. It may be useful for some users who wish to use pip
to install everything they need.
To test:
./bootstrap && (cd pkgs/sageconf_pypi/ && tox r v v v)
This test using tox
uses .sage
in an isolated directory within .tox
instead of in your home directory. At the end of the tox run, the installed sageconfig
script is executed; it prints the configuration variables:
[10639] /Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/pkgs/sageconf_pypi$ /Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/pkgs/sageconfpypi/.tox/python/bin/sageconfig VERSION=9.3.beta7 MAXIMA=/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.beta7cpython39darwin/local/bin/maxima ARB_LIBRARY=arb SAGE_NAUTY_BINS_PREFIX= CBLAS_PC_MODULES=cblas MATHJAX_DIR=/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.beta7cpython39darwin/local/share/mathjax THREEJS_DIR=/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.beta7cpython39darwin/local/share/threejs SAGE_LOCAL=/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.beta7cpython39darwin/local SAGE_ROOT=/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.beta7cpython39darwin python finish: runtest after 0.17 seconds
To build a source distribution:
./sage sh c build/pkgs/sage_conf/spkgsrc
(The source distribution has been uploaded to https://pypi.org/project/sageconf/)
Followup steps:
 Make SAGE_ROOT configurable by the user (by passing
prefix
tosetup.py
).  Optionally, it would create a venv that is used for building wheels (#30527).
 Using #29013, it could support several python versions
 #30036: Make the build more incremental: On installation of
sage_conf
, only runconfigure
but do not install anything; rather, a Python entry point can be used by Python packages such assagemathstandard
to install packages if present.
Change History (126)
comment:1 Changed 18 months ago by
 Description modified (diff)
 Milestone changed from sage9.1 to sagewishlist
comment:2 Changed 13 months ago by
 Description modified (diff)
 Milestone changed from sagewishlist to sage9.3
 Summary changed from Package sage_local to pipinstallable package sage_bootstrap
comment:3 Changed 13 months ago by
 Description modified (diff)
comment:4 Changed 13 months ago by
 Dependencies set to #30025
 Description modified (diff)
comment:5 Changed 13 months ago by
 Branch set to u/mkoeppe/pip_installable_package_sage_bootstrap
comment:6 Changed 13 months ago by
 Commit set to 6e28ae56c3eab514bd350182640e00489dc81ab0
 Description modified (diff)
comment:7 Changed 13 months ago by
 Commit changed from 6e28ae56c3eab514bd350182640e00489dc81ab0 to a6373b762d6a35cf51e48a005d9a5b04bcc621f9
Branch pushed to git repo; I updated commit sha1. New commits:
a6373b7  build/setup.py: Add sage_root as package_data

comment:8 Changed 13 months ago by
 Commit changed from a6373b762d6a35cf51e48a005d9a5b04bcc621f9 to b0ddab043224bf868bc0753ed2afbf1e3ad9404c
comment:9 Changed 13 months ago by
 Dependencies changed from #30025 to #30025, #29850
comment:10 Changed 13 months ago by
 Commit changed from b0ddab043224bf868bc0753ed2afbf1e3ad9404c to 60642d345c9961990f0e126be9af58ab3f893143
comment:11 Changed 13 months ago by
 Description modified (diff)
comment:12 Changed 13 months ago by
 Dependencies changed from #30025, #29850 to #30025
comment:13 Changed 13 months ago by
 Commit changed from 60642d345c9961990f0e126be9af58ab3f893143 to 8006b6f9d6a2b531f067b42d416f00d591d63aaf
Branch pushed to git repo; I updated commit sha1. New commits:
8006b6f  Add sage_bootstrap.build

comment:14 Changed 13 months ago by
 Work issues set to sagesystempython in installed sage_bootstrap
comment:15 Changed 13 months ago by
comment:16 Changed 11 months ago by
 Dependencies changed from #30025 to #30025, #30527
 Description modified (diff)
comment:17 Changed 11 months ago by
 Dependencies changed from #30025, #30527 to #30025, #30527, #29013
 Description modified (diff)
comment:18 Changed 9 months ago by
 Dependencies changed from #30025, #30527, #29013 to #30527, #29013
 Description modified (diff)
comment:19 Changed 8 months ago by
 Dependencies #30527, #29013 deleted
comment:20 Changed 8 months ago by
 Commit changed from 8006b6f9d6a2b531f067b42d416f00d591d63aaf to 9293ae1d878c7fa24a3098687c49757027be361b
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
a1ad0be  Add sage_root to sage_bootstrap

8695368  build/setup.py: Add sage_root as package_data

cc93eca  Working sdist and install

45c0ebe  Simplify directory structure

62d4f6c  build/setup.py: Reuse configured sage root

7d705e3  Readd build/sage_root

44e3224  build/setup.py: Install sage_conf.py, sageenvconfig

9293ae1  Add sage_bootstrap.build

comment:21 Changed 8 months ago by
 Commit changed from 9293ae1d878c7fa24a3098687c49757027be361b to 3f243e19bd975f4c71b34a049f90d0ff2097a8d0
comment:22 Changed 8 months ago by
 Keywords sd111 added
comment:23 Changed 7 months ago by
 Description modified (diff)
 Summary changed from pipinstallable package sage_bootstrap to pipinstallable packages sage_bootstrap, sage_conf
comment:24 Changed 7 months ago by
 Dependencies set to #30913
comment:25 Changed 6 months ago by
 Work issues changed from sagesystempython in installed sage_bootstrap to sagesystempython in installed sage_bootstrap. Rework it so that sage_conf is the package running configure.
comment:26 Changed 6 months ago by
 Commit changed from 3f243e19bd975f4c71b34a049f90d0ff2097a8d0 to b7cd894f49dd4f39824f421166faf281fb64d641
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
7da13f4  Add sage_root to sage_bootstrap

9ea9d80  build/setup.py: Add sage_root as package_data

3f5c99c  Working sdist and install

1f7647f  Simplify directory structure

f59d077  build/setup.py: Reuse configured sage root

4a18b98  Readd build/sage_root

8e44d28  build/setup.py: Install sage_conf.py, sageenvconfig

86e21a1  Add sage_bootstrap.build

47f5e95  build/sage_root: Remove outdated symlink

b7cd894  build/setup.py: Use configure withpython=...

comment:27 Changed 6 months ago by
 Commit changed from b7cd894f49dd4f39824f421166faf281fb64d641 to 44b873b3023f487e0188554b1d353520db41f884
comment:28 Changed 6 months ago by
 Dependencies changed from #30913 to #30913, #31362
 Description modified (diff)
 Summary changed from pipinstallable packages sage_bootstrap, sage_conf to pipinstallable version of package sage_conf
comment:29 Changed 6 months ago by
 Commit changed from 44b873b3023f487e0188554b1d353520db41f884 to 8ee90871c44a31ebf3a48932567475ae1bcec9b5
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
8ee9087  Put both versions of sage_conf sources into subdirectories of SAGE_ROOT/src/pkgs/

comment:30 Changed 6 months ago by
 Commit changed from 8ee90871c44a31ebf3a48932567475ae1bcec9b5 to 47e182d81ec5fa6fa810fd30316bdf42047b067c
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
47e182d  Put both versions of sage_conf sources into subdirectories of SAGE_ROOT/src/pkgs/

comment:31 Changed 6 months ago by
 Dependencies changed from #30913, #31362 to #30913, #31362, #31357
comment:32 Changed 6 months ago by
 Commit changed from 47e182d81ec5fa6fa810fd30316bdf42047b067c to b2b785b2b9ae6194a7d15f2f1b1ac694ed388e84
Branch pushed to git repo; I updated commit sha1. New commits:
55bc2aa  m4/sage_spkg_collect.m4: Generate install tree information for build/make/Makefile

d7dc9a2  build/make/Makefile.in: New targets allsagelocal, allsagevenv

27b589b  Merge branch 't/30912/sagelib__update_metadata_for_pypi_deployment' into t/31362/make_all_sage_local__make_all_sage_venv

8796008  Put pynac into SAGE_LOCAL, sagelib into SAGE_VENV

d6831b1  build/make/Makefile.in: Add allbuildlocal, allbuildvenv, which include dependency on toolchain

61f6ba6  Makefile: Add toplevel targets buildlocal, buildvenv

b2b785b  Put both versions of sage_conf sources into subdirectories of SAGE_ROOT/src/pkgs/

comment:33 Changed 6 months ago by
 Description modified (diff)
 Work issues sagesystempython in installed sage_bootstrap. Rework it so that sage_conf is the package running configure. deleted
comment:34 Changed 6 months ago by
 Dependencies changed from #30913, #31362, #31357 to #30913, #31362, #31357, #31321
 Description modified (diff)
comment:35 Changed 6 months ago by
 Commit changed from b2b785b2b9ae6194a7d15f2f1b1ac694ed388e84 to b8b8a0607d3a9d7ad05df8c8e306c41c82a05e24
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
b8b8a06  Put both versions of sage_conf sources into subdirectories of SAGE_ROOT/src/pkgs/

comment:36 Changed 6 months ago by
 Commit changed from b8b8a0607d3a9d7ad05df8c8e306c41c82a05e24 to ff7f110f406e3a88ae655b7c77b8f0f93d213006
Branch pushed to git repo; I updated commit sha1. New commits:
ff7f110  build/pkgs/sage_conf/installrequires.txt: New

comment:37 Changed 6 months ago by
 Commit changed from ff7f110f406e3a88ae655b7c77b8f0f93d213006 to 072b522e2e10bdcec729f72dcaa82908154864b7
Branch pushed to git repo; I updated commit sha1. New commits:
072b522  Determine SAGE_ROOT from sage version and python version

comment:38 Changed 6 months ago by
 Commit changed from 072b522e2e10bdcec729f72dcaa82908154864b7 to fa4cbfa8bf933c41c1a6d0d649fb0fb3b5030aa1
Branch pushed to git repo; I updated commit sha1. New commits:
fa4cbfa  Run .homebrewbuildenv and set 'make j...' automatically

comment:39 Changed 6 months ago by
 Description modified (diff)
 Priority changed from major to critical
 Status changed from new to needs_review
 Summary changed from pipinstallable version of package sage_conf to pipinstallable version of package sage_conf  installs nonPython bits of the Sage distribution in ~/.sage/
comment:40 Changed 6 months ago by
 Cc jhpalmieri dimpase fbissey added
 Description modified (diff)
comment:41 Changed 6 months ago by
 Commit changed from fa4cbfa8bf933c41c1a6d0d649fb0fb3b5030aa1 to fa9485cfc9a47aec79c77f73575c56c56c197666
Branch pushed to git repo; I updated commit sha1. New commits:
fa9485c  Remove sage_bootstrap.build (not needed for this ticket)

comment:42 Changed 6 months ago by
 Description modified (diff)
comment:43 Changed 6 months ago by
 Commit changed from fa9485cfc9a47aec79c77f73575c56c56c197666 to 7bdc52452f725002185d58d95735f7ee191ce469
Branch pushed to git repo; I updated commit sha1. New commits:
7bdc524  build/pkgs/sage_conf/src: Restore symlink

comment:44 Changed 6 months ago by
 Commit changed from 7bdc52452f725002185d58d95735f7ee191ce469 to 76d151e35870c8e52a2423058c497e3e6252b517
Branch pushed to git repo; I updated commit sha1. New commits:
76d151e  build/bin/writedockerfile.sh: Fix up docker tests

comment:45 Changed 6 months ago by
 Commit changed from 76d151e35870c8e52a2423058c497e3e6252b517 to 1a8cadc8975ce9d7d662678c7a8019202c084b46
Branch pushed to git repo; I updated commit sha1. New commits:
1a8cadc  src/pkgs/sage_confpypi/setup.cfg: Under version control, not gitignored

comment:46 Changed 5 months ago by
 Commit changed from 1a8cadc8975ce9d7d662678c7a8019202c084b46 to 31cadddccf1901b71c194aa1617929b738cf131d
Branch pushed to git repo; I updated commit sha1. New commits:
31caddd  Merge tag '9.3.beta8' into t/29039/pip_installable_package_sage_bootstrap

comment:47 Changed 5 months ago by
 Commit changed from 31cadddccf1901b71c194aa1617929b738cf131d to 48d2a2bf872e15804f7125e39dc3abb19d1eaf49
comment:48 Changed 5 months ago by
 Dependencies changed from #30913, #31362, #31357, #31321 to #30913
comment:49 Changed 5 months ago by
 Dependencies changed from #30913 to #30913, #30383
comment:50 Changed 5 months ago by
 Commit changed from 48d2a2bf872e15804f7125e39dc3abb19d1eaf49 to 5dbd712fa100252e2472a4dd33095bbe1a4c5a7a
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
82d8030  m4/sage_spkg_enable.m4: Cosmetic change to help strings

3e92998  configure disablenotebook: Fix up what packages are disabled

74a83fc  build/pkgs/matplotlib/dependencies: Undo removal of tornado

93d31b9  Remove use of unicode superscripts in configure help

18fbb85  m4/sage_spkg_collect.m4: Fix description of SAGE_OPTIONAL_CLEANED_PACKAGES

c3e4093  Rename SAGE_OPTIONAL_CLEANED_PACKAGES to SAGE_OPTIONAL_UNINSTALLED_PACKAGES

4916415  Merge tag '9.3.beta9' into t/30383/new_package_type__optional_enabled_by_default

a35428e  Merge #30383

e098133  src/pkgs/sage_conf/sage_conf.py.in: Add SAGE_SPKG_WHEELS

5dbd712  src/pkgs/sage_confpypi/setup.py: Use 'configure disablenotebook', run 'make build' so that wheels are built

comment:51 Changed 5 months ago by
 Commit changed from 5dbd712fa100252e2472a4dd33095bbe1a4c5a7a to 100ff515319f6f7fbe66487a5bb35fc34cd3b07f
comment:52 Changed 5 months ago by
 Commit changed from 100ff515319f6f7fbe66487a5bb35fc34cd3b07f to a4f75b86bf2c70bbc4edb7f0d5ee558ccf75e1b0
comment:53 Changed 4 months ago by
 Commit changed from a4f75b86bf2c70bbc4edb7f0d5ee558ccf75e1b0 to ac0f563ba1fff7ed111499bead102573d83dc1c8
Branch pushed to git repo; I updated commit sha1. New commits:
66c2987  build/pkgs/{sage_sws2rst,sage_docbuild}/installrequires.txt: New

7f8ec6c  build/pkgs/sagelib/src/tox.ini: Add factor sitepackages

0283da5  build/make/Makefile.in: Add wheel, setuptools_wheel to PYTHON_TOOLCHAIN to make sure that PEP 517 packages have a complete build system

f720722  build/pkgs/sagelib/src/tox.ini: Add factor nobuildisolation

c451b31  src/setup.cfg.m4 (install_requires): Add sage_conf

6700223  Merge tag '9.3.rc0' into t/30913/sagelib__add_setup_cfg__install_requires_

ac0f563  Merge branch 't/30913/sagelib__add_setup_cfg__install_requires_' into t/29039/pip_installable_package_sage_bootstrap

comment:54 Changed 4 months ago by
 Dependencies changed from #30913, #30383 to #30913
comment:55 Changed 4 months ago by
 Description modified (diff)
comment:56 Changed 4 months ago by
 Description modified (diff)
comment:57 followup: ↓ 60 Changed 4 months ago by
what does the change to build/pkgs/sagelib/src/MANIFEST.in
mean? is it becoming a symbolic link?
comment:58 followup: ↓ 62 Changed 4 months ago by
after merging 9.3.rc2 in, I get an error (is it from sh: line 0: .: .homebrewbuildenv: file not found
, cf below ?):
$ ./bootstrap && (cd src/pkgs/sage_confpypi/ && tox v v v) ... warning: no previouslyincluded files matching '__pycache__' found anywhere in distribution warning: no previouslyincluded files matching '*.pyc' found anywhere in distribution writing manifest file '/home/scratch2/dimpase/tmp/pipmodernmetadata27nf1gz1/sage_conf.egginfo/SOURCES.txt' creating '/home/scratch2/dimpase/tmp/pipmodernmetadata27nf1gz1/sage_conf.distinfo' Preparing wheel metadata ... done Source in /home/scratch2/dimpase/tmp/pipreqbuild0rrkid61 has version 9.3rc2, which satisfies requirement sageconf==9.3rc2 from file:///home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3rc2.zip Removed sageconf==9.3rc2 from file:///home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3rc2.zip from build tracker '/home/scratch2/dimpase/tmp/pipreqtrackerbla_u_md' Created temporary directory: /home/scratch2/dimpase/tmp/pipunpackrwlm13mn Building wheels for collected packages: sageconf Created temporary directory: /home/scratch2/dimpase/tmp/pipwheelk4duer3n Destination directory: /home/scratch2/dimpase/tmp/pipwheelk4duer3n Running command /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_vendor/pep517/_in_process.py build_wheel /home/scratch2/dimpase/tmp/tmpdhib0roj running bdist_wheel running build running build_py sh: line 0: .: .homebrewbuildenv: file not found Running cd /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu && . .homebrewbuildenv 2>&1; : && ./configure prefix=/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu/local withpython=/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python withsystempython3=force disablenotebook disablesagelib error: configure failed Building wheel for sageconf (PEP 517) ... error ERROR: Failed building wheel for sageconf Failed to build sageconf ERROR: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly Exception information: Traceback (most recent call last): File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_internal/cli/base_command.py", line 189, in _main status = self.run(options, args) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_internal/cli/req_command.py", line 178, in wrapper return func(self, options, args) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_internal/commands/install.py", line 361, in run ", ".join(pep517_build_failure_names) pip._internal.exceptions.InstallationError: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly Removed build tracker: '/home/scratch2/dimpase/tmp/pipreqtrackerbla_u_md' python finish: installpkg after 3.39 seconds python start: envreport setting PATH=/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin:/users/dimpase/perl5/bin:/users/dimpase/bin:/usr/libexec/python3sphinx:/usr/lib64/qt3.3/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/users/dimpase/.cabal/bin:/auto/users/dimpase/bin /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi$ /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python m pip freeze >/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/log/python3.log python finish: envreport after 0.20 seconds python installed: _________________________________________________________________________________________________ summary __________________________________________________________________________________________________ ERROR: python: InvocationError for command '/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install nodeps U v /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3rc2.zip' (exited with code 1) cleanup /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3rc2.zip
comment:59 Changed 4 months ago by
 Reviewers set to Dima Pasechnik
 Status changed from needs_review to needs_work
this is on Fedora 30, in case it matters
comment:60 in reply to: ↑ 57 Changed 4 months ago by
Replying to dimpase:
what does the change to
build/pkgs/sagelib/src/MANIFEST.in
mean? is it becoming a symbolic link?
Yes, it was duplicated with SAGE_ROOT/src/MANIFEST.in
, and I replaced it with a symlink
comment:61 Changed 4 months ago by
 Commit changed from ac0f563ba1fff7ed111499bead102573d83dc1c8 to 1b03fa9a4f14439d0dbc3bd1af194ab25fa9eeb8
comment:62 in reply to: ↑ 58 Changed 4 months ago by
Replying to dimpase:
after merging 9.3.rc2 in, I get an error (is it from
sh: line 0: .: .homebrewbuildenv: file not found
, cf below ?):$ ./bootstrap && (cd src/pkgs/sage_confpypi/ && tox v v v) ...
Thanks for testing! Is this running in a docker container? Not clear to me why the file would be missing. Anyway I have cherrypicked a related change from #31396 and made another change  please take a look if this gets farther
comment:63 Changed 4 months ago by
 Status changed from needs_work to needs_review
comment:64 Changed 4 months ago by
no, still the same error:
... Created temporary directory: /home/scratch2/dimpase/tmp/pipunpack169y8koj Building wheels for collected packages: sageconf Created temporary directory: /home/scratch2/dimpase/tmp/pipwheelexb6ynmx Destination directory: /home/scratch2/dimpase/tmp/pipwheelexb6ynmx Running command /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_vendor/pep517/_in_process.py build_wheel /home/scratch2/dimpase/tmp/tmpvehv3h0s running bdist_wheel running build running build_py sh: line 0: .: .homebrewbuildenv: file not found Running cd /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu && (. .homebrewbuildenv 2>&1  :) && ./configure prefix=/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu/local withpython=/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python withsystempython3=force disablenotebook disablesagelib error: configure failed Building wheel for sageconf (PEP 517) ... error ERROR: Failed building wheel for sageconf Failed to build sageconf ERROR: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly Exception information: Traceback (most recent call last): File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_internal/cli/base_command.py", line 189, in _main status = self.run(options, args) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_internal/cli/req_command.py", line 178, in wrapper return func(self, options, args) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pip/_internal/commands/install.py", line 361, in run ", ".join(pep517_build_failure_names) pip._internal.exceptions.InstallationError: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly Removed build tracker: '/home/scratch2/dimpase/tmp/pipreqtracker_6q2pk49' python finish: installpkg after 3.35 seconds python start: envreport setting PATH=/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin:/users/dimpase/perl5/bin:/users/dimpase/bin:/usr/libexec/python3sphinx:/usr/lib64/qt3.3/bin:/usr/share/Modules/bin:/usr/lib64/ccache:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/users/dimpase/.cabal/bin:/auto/users/dimpase/bin /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi$ /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python m pip freeze >/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/log/python9.log python finish: envreport after 0.20 seconds python installed: __________________________________________________________________________ summary ___________________________________________________________________________ ERROR: python: InvocationError for command '/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install nodeps U v /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3rc2.zip' (exited with code 1)
comment:65 Changed 4 months ago by
 Status changed from needs_review to needs_work
comment:66 Changed 4 months ago by
Could you show what's in /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu
?
comment:67 Changed 4 months ago by
$ ls la /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu total 1196 drwxrxrx 6 dimpase staff 4096 Apr 9 20:24 . drwxrxrx 4 dimpase staff 4096 Apr 9 20:24 .. rwxrxrx 1 dimpase staff 81 Apr 9 20:24 bootstrap drwxrxrx 7 dimpase staff 4096 Apr 9 20:24 build drwxrxrx 2 dimpase staff 4096 Apr 9 20:24 config rwxrxrx 1 dimpase staff 1126829 Apr 9 20:24 configure rwrr 1 dimpase staff 18020 Apr 9 20:24 configure.ac rwrr 1 dimpase staff 1602 Apr 9 20:24 .homebrewbuildenv drwxrxrx 2 dimpase staff 4096 Apr 9 20:24 m4 rwrr 1 dimpase staff 9744 Apr 9 20:24 Makefile rwrr 1 dimpase staff 20890 Apr 9 20:24 README.md drwxrxrx 4 dimpase staff 4096 Apr 9 20:24 src rwrr 1 dimpase staff 51 Apr 9 20:24 VERSION.txt
comment:68 Changed 4 months ago by
 Commit changed from 1b03fa9a4f14439d0dbc3bd1af194ab25fa9eeb8 to 278c36c6caf9dfa8e7d33e0b45309615ee13a163
Branch pushed to git repo; I updated commit sha1. New commits:
b3b54b7  Merge tag '9.3.rc2' into t/29039/pip_installable_package_sage_bootstrap

ec10bc9  src/pkgs/sage_confpypi/setup.py: Fix for shell behavior on fedora

278c36c  src/pkgs/sage_confpypi/setup.py: Use 'configure enablebuildasroot' so it works in containers too

comment:69 Changed 4 months ago by
 Status changed from needs_work to needs_review
comment:70 followup: ↓ 72 Changed 4 months ago by
the dependent ticket is not merged in the branch, any reason for this?
comment:71 Changed 4 months ago by
 Commit changed from 278c36c6caf9dfa8e7d33e0b45309615ee13a163 to 94b8d3c7223785fab507ecbe38f9c16fb44a46de
comment:72 in reply to: ↑ 70 Changed 4 months ago by
Replying to dimpase:
the dependent ticket is not merged in the branch, any reason for this?
Thanks for catching this, the latest version was not merged
comment:73 Changed 4 months ago by
Here on the ticket I am creating things in src/pkgs/
but in #31577 I have a different proposal for the source layout, comments welcome
comment:74 Changed 4 months ago by
now I get a failure in building numpy, if I run
./bootstrap && (cd src/pkgs/sage_confpypi/ && tox v v v)
after distclean.
Uninstalling 'numpy' with legacy uninstaller Package 'numpy' is currently not installed Traceback (most recent call last): File "../lapack_conf.py", line 3, in <module> import pkgconfig, os ModuleNotFoundError: No module named 'pkgconfig'
and also weird error with cysignal:
checking for std::atomic with OpenMP in C++ code... yes configure: creating ./config.status config.status: creating src/cysignals/signals.pxd config.status: creating src/config.h config.status: creating src/cysignals/cysignals_config.h configure: WARNING: unrecognized options: disablemaintainermode, disabledependencytracking Installing cysignals1.10.3 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fda2cad2310>, 'Connection to 192.0.2.0 timed out. (connect timeout=15)')': /simple/cython/ WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fda2cad2690>, 'Connection to 192.0.2.0 timed out. (connect timeout=15)')': /simple/cython/ WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fda2cad2890>, 'Connection to 192.0.2.0 timed out. (connect timeout=15)')': /simple/cython/ WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fda2cad2ad0>, 'Connection to 192.0.2.0 timed out. (connect timeout=15)')': /simple/cython/ WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fda2cad2d10>, 'Connection to 192.0.2.0 timed out. (connect timeout=15)')': /simple/cython/ ERROR: Could not find a version that satisfies the requirement Cython>=0.28 ERROR: No matching distribution found for Cython>=0.28 Traceback (most recent call last): File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/setuptools/installer.py", line 75, in fetch_build_egg subprocess.check_call(cmd) File "/usr/lib64/python3.7/subprocess.py", line 363, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu/local/bin/python3', 'm', 'pip', 'disablepipversioncheck', 'wheel', 'nodeps', 'w', '/home/scratch2/dimpase/tmp/tmp772nb9rh', 'quiet', 'Cython>=0.28']' returned nonzero exit status 1. The above exception was the direct cause of the following exception: Traceback (most recent call last): File "setup.py", line 152, in <module> cmdclass=dict(build=build, bdist_egg=no_egg), File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/setuptools/__init__.py", line 152, in setup _install_setup_requires(attrs) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/setuptools/__init__.py", line 147, in _install_setup_requires dist.fetch_build_eggs(dist.setup_requires) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/setuptools/dist.py", line 724, in fetch_build_eggs replace_conflicting=True, File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pkg_resources/__init__.py", line 768, in resolve replace_conflicting=replace_conflicting File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pkg_resources/__init__.py", line 1051, in best_match return self.obtain(req, installer) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/pkg_resources/__init__.py", line 1063, in obtain return installer(requirement) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/setuptools/dist.py", line 780, in fetch_build_egg return fetch_build_egg(self, req) File "/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/lib/python3.7/sitepackages/setuptools/installer.py", line 77, in fetch_build_egg raise DistutilsError(str(e)) from e distutils.errors.DistutilsError: Command '['/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu/local/bin/python3', 'm', 'pip', 'disablepipversioncheck', 'wheel', 'nodeps', 'w', '/home/scratch2/dimpase/tmp/tmp772nb9rh', 'quiet', 'Cython>=0.28']' returned nonzero exit status 1. Error: could not determine package name
dependencies issue?
comment:75 Changed 4 months ago by
Note that make distclean
does not clean stuff in .tox
, so if you want to start from scratch, use tox r
comment:76 Changed 4 months ago by
still, after successful make build
, running the command in the ticket description ends with
Error building Sage. The following package(s) may have failed to build (not necessarily during this run of 'make allbuild'): * package: numpy1.19.5 last build time: Apr 11 22:40 log file: /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu/logs/pkgs/numpy1.19.5.log build directory: /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu/local/var/tmp/sage/build/numpy1.19.5
and in the numpy log I see
Uninstalling 'numpy' with legacy uninstaller Package 'numpy' is currently not installed Traceback (most recent call last): File "../lapack_conf.py", line 3, in <module> import pkgconfig, os ModuleNotFoundError: No module named 'pkgconfig' real 0m0.064s user 0m0.049s sys 0m0.010s ************************************************************************ Error installing package numpy1.19.5
a race condition, pkgconfig
is not installed yet?
comment:77 Changed 4 months ago by
and this
$ ./sage python Python 3.7.7 (default, Mar 13 2020, 21:39:43) [GCC 9.2.1 20190827 (Red Hat 9.2.11)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pkgconfig >>>
is OK. Is something weird happening with the Python while building numpy?
comment:78 Changed 4 months ago by
Oh, I see. The script lapack_conf.py
calls sagebootstrappython
, which is not Sage's python in this case, it's a python2.
clpc171[/home/scratch2/dimpase/sage/sage]$ ./sage buildsh Starting subshell with Sage environment variables set. Don't forget to exit when you are done. Beware: * Do not do anything with other copies of Sage on your system. * Do not use this for installing Sage packages using "sage i" or for running "make" at Sage's root directory. These should be done outside the Sage shell. Bypassing shell configuration files... Note: SAGE_ROOT=/home/scratch2/dimpase/sage/sage (sagebuildsh) dimpase@clpc171:sage$ sagebootstrappython Python 2.7.18 (default, Apr 21 2020, 18:49:31) [GCC 9.3.1 20200408 (Red Hat 9.3.12)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>>
comment:79 Changed 4 months ago by
 Status changed from needs_review to needs_work
comment:80 Changed 4 months ago by
That's really something that we need to fix in a separate ticket!
comment:81 followup: ↓ 83 Changed 4 months ago by
No, actually spkginstall.in
calls python3
.
comment:82 followup: ↓ 85 Changed 4 months ago by
You are interacting with the wrong SAGE_ROOT, by the way. The relevant SAGE_ROOT
is /home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu
. What's happening in /home/scratch2/dimpase/sage/sage
is irrelevant.
comment:83 in reply to: ↑ 81 Changed 4 months ago by
Replying to mkoeppe:
No, actually
spkginstall.in
callspython3
.
well, it calls the failing lapack_conf.py
, which prescribes a different Python.
comment:84 Changed 4 months ago by
See #31663, where I explain this in more detail
comment:85 in reply to: ↑ 82 Changed 4 months ago by
Replying to mkoeppe:
You are interacting with the wrong SAGE_ROOT, by the way. The relevant
SAGE_ROOT
is/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu
. What's happening in/home/scratch2/dimpase/sage/sage
is irrelevant.
OK, so I guess here is the test showing that pkgconfig
module is not there
(I cd
to the said SAGE_ROOT
and run ./local/bin/python3
there):
clpc171[/home/scratch2/dimpase/sage/sage/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3.rc2cpython37mx86_64linuxgnu]$ ./local/bin/python3 Python 3.7.7 (default, Mar 13 2020, 21:39:43) [GCC 9.2.1 20190827 (Red Hat 9.2.11)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import pkgconfig Traceback (most recent call last): File "<stdin>", line 1, in <module> ModuleNotFoundError: No module named 'pkgconfig'
(and the same import works for the "usual" SAGE_ROOT just fine)
comment:86 Changed 4 months ago by
If I pipinstall pkgconfig
and cython
in the correct SAGE_ROOT, then tox run gets through numpy install just fine, but fails later in a similar way on pplpy.
I guess dependency resolution is shot, or there is a race somewhere, leading to wrong order of installations.
comment:87 Changed 4 months ago by
Thanks for testing! This is very strange...
comment:88 Changed 4 months ago by
 Milestone changed from sage9.3 to sage9.4
comment:89 Changed 3 months ago by
 Commit changed from 94b8d3c7223785fab507ecbe38f9c16fb44a46de to df2f8d11a2d204b5e5197b968ab1bb8e62486508
Branch pushed to git repo; I updated commit sha1. New commits:
df2f8d1  Merge tag '9.3' into t/29039/pip_installable_package_sage_bootstrap

comment:90 Changed 3 months ago by
 Status changed from needs_work to needs_review
It may be worth testing whether at least the built source distribution (https://pypi.org/project/sagemathstandard/) works for you as advertised.
comment:91 followup: ↓ 93 Changed 3 months ago by
it seems to parallelize make
too aggressively. Just had to reboot a machine I tried this on.
Namely, top
shows a massive number of make
instances launched, hundreds. The log is full of
make[20]: warning: j4 forced in submake: resetting jobserver mode. make j4 build/make/Makefile stop
Also, the 1st attempt to run tox v v v
failed with a message like "packages PEP <something>
are not directly installable" (or something like this). I just started it again, and it went on.
comment:92 Changed 3 months ago by
 Description modified (diff)
comment:93 in reply to: ↑ 91 Changed 3 months ago by
Replying to dimpase:
it seems to parallelize
make
too aggressively. Just had to reboot a machine I tried this on.
This sounds more like an infinite "make" recursion. Could you remove the setting of j
from src/pkgs/sage_confpypi/setup.py
(and from your environment) and try what it does then?
comment:94 Changed 3 months ago by
I still quickly get make
completely taking over the machine:
$ ps def  grep make  wc l 1797
even after

src/pkgs/sage_confpypi/setup.py
diff git a/src/pkgs/sage_confpypi/setup.py b/src/pkgs/sage_confpypi/setup.py index b7646224d6..8ee7211104 100644
a b class build_py(distutils_build_py): 52 52 # but then a user could use "make buildvenv" to build compatible wheels for all Python packages. 53 53 # TODO: A target to only build wheels of tricky packages 54 54 # (that use native libraries shared with other packages). 55 SETMAKE = 'if [ z "$MAKE" ]; then export MAKE="make j$(PATH=build/bin:$PATH build/bin/sagebuildnumthreads  cut d" " f 2)"; fi'55 SETMAKE = 'if [ z "$MAKE" ]; then export MAKE="make"; fi' 56 56 cmd = f'cd {SAGE_ROOT} && {SETENV} && {SETMAKE} && $MAKE V=0 build' 57 57 if os.system(cmd) != 0: 58 58 raise DistutilsSetupError("make buildlocal failed")
no more warning: j4 forced in submake: resetting jobserver mode
seen, but that's not the root cause. This is with make 4.3.
comment:95 Changed 3 months ago by
It could be that some wrong links are created by git; if I try to pull this branch I get
CONFLICT (directory/file): There is a directory with name build/pkgs/sage_conf/src in HEAD. Adding build/pkgs/sage_conf/src as build/pkgs/sage_conf/src~df2f8d11a2d204b5e5197b968ab1bb8e62486508 Automatic merge failed; fix conflicts and then commit the result.
and perhaps I didn't fix conflicts correctly.
It would be good to have something rebased on the develop branch here.
comment:96 Changed 3 months ago by
OK, on a clean tree checkout I don't get the make issues. Apologies.
comment:97 Changed 3 months ago by
./bootstrap && (cd src/pkgs/sage_confpypi/ && tox r v v v)
seems to work, although I had to restart tox r v v v
at some moment. Retrying after distclean, to see if I can reproduce this.
I'm getting
make noprintdirectory polytopes_dbnodeps sagelogger p 'SAGE_CHECK=no sagespkg polytopes_db20170220.p0' '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/polytopes_db20170220.p0.log' [polytopes_db20170220.p0] installing. Log file: /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/polytopes_db20170220.p0.log [polytopes_db20170220.p0] successfully installed. make noprintdirectory singularnodeps sagelogger p 'SAGE_CHECK=no sagespkg singular4.2.0p1+20210406+sage' '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/singular4.2.0p1+20210406+sage.log' [singular4.2.0p1+20210406+sage] installing. Log file: /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/singular4.2.0p1+20210406+sage.log [palp2.11] successfully installed. make noprintdirectory ratpointsnodeps make[3]: *** No rule to make target '../pkgs/sage_conf/src/bin/sageenvconfig', needed by '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/local/var/lib/sage/installed/sage_conf9.3'. Stop. make[3]: *** Waiting for unfinished jobs.... sagelogger p 'SAGE_CHECK=no sagespkg ratpoints2.1.3.p5' '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/ratpoints2.1.3.p5.log' [ratpoints2.1.3.p5] installing. Log file: /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/ratpoints2.1.3.p5.log [ratpoints2.1.3.p5] successfully installed. [ecl21.2.1] successfully installed. [gap4.11.0.p1] successfully installed. [singular4.2.0p1+20210406+sage] successfully installed. make[2]: *** [Makefile:2150: allbuild] Error 2 make[2]: Leaving directory '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/build/make' real 9m41.134s user 24m26.005s sys 1m57.351s *************************************************************** Error building Sage. The following package(s) may have failed to build (not necessarily during this run of 'make allbuild'): It is safe to delete any log files and build directories, but they contain information that is helpful for debugging build problems. WARNING: If you now run 'make' again, the build directory of the same version of the package will, by default, be deleted. Set the environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this. make[1]: *** [Makefile:39: allbuild] Error 1 make[1]: Leaving directory '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu' make: *** [Makefile:16: build] Error 2 Running cd /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu && (. ./.homebrewbuildenv 2> /dev/null  :) && ./configure prefix=/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/local withpython=/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python enablebuildasroot withsystempython3=force disablenotebook disablesagelib error: make buildlocal failed Building wheel for sageconf (PEP 517) ... error ERROR: Failed building wheel for sageconf Failed to build sageconf ERROR: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly Exception information: Traceback (most recent call last): File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/cli/base_command.py", line 189, in _main status = self.run(options, args) File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/cli/req_command.py", line 178, in wrapper return func(self, options, args) File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/commands/install.py", line 358, in run raise InstallationError( pip._internal.exceptions.InstallationError: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly WARNING: You are using pip version 21.0.1; however, version 21.1.1 is available. You should consider upgrading via the '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install upgrade pip' command. Removed build tracker: '/tmp/pipreqtrackeredpp7niq' ERROR: invocation failed (exit code 1) python finish: installpkg /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3.zip after 622.22 seconds _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ summary __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ERROR: python: InvocationError for command /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install existsaction w v .tox/.tmp/package/1/sage_conf9.3.zip (exited with code 1) cleanup /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3.zip
At this point I can just start tox r v v v
again, and it happily continues.
Not sure whether the final result is correct.
Anyway, the error isn't good.
Looks like a bit of an error in a make rule, it tries to make ../pkgs/sage_conf/src/bin/sageenvconfig
.
comment:98 Changed 3 months ago by
After the restart, I see again
make[3]: *** No rule to make target '../pkgs/sage_conf/src/bin/sageenvconfig', needed by '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/local/var/lib/sage/installed/sage_conf9.3'. Stop. make[3]: *** Waiting for unfinished jobs....
but it does not lead to (immediate) termination.
comment:99 Changed 3 months ago by
after a while it then errors out with already mentioned
Failed to build sageconf ERROR: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly
in more detail:
... [singular4.2.0p1+20210406+sage] successfully installed. make[2]: *** [Makefile:2150: allbuild] Error 2 make[2]: Leaving directory '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/build/make' real 10m35.056s user 24m37.840s sys 2m0.611s *************************************************************** Error building Sage. The following package(s) may have failed to build (not necessarily during this run of 'make allbuild'): It is safe to delete any log files and build directories, but they contain information that is helpful for debugging build problems. WARNING: If you now run 'make' again, the build directory of the same version of the package will, by default, be deleted. Set the environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this. make[1]: *** [Makefile:39: allbuild] Error 1 make[1]: Leaving directory '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu' make: *** [Makefile:16: build] Error 2 Running cd /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu && (. ./.homebrewbuildenv 2> /dev/null  :) && ./configure prefix=/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/local withpython=/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python enablebuildasroot withsystempython3=force disablenotebook disablesagelib error: make buildlocal failed Building wheel for sageconf (PEP 517) ... error ERROR: Failed building wheel for sageconf Failed to build sageconf ERROR: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly Exception information: Traceback (most recent call last): File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/cli/base_command.py", line 189, in _main status = self.run(options, args) File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/cli/req_command.py", line 178, in wrapper return func(self, options, args) File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/commands/install.py", line 358, in run raise InstallationError( pip._internal.exceptions.InstallationError: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly WARNING: You are using pip version 21.0.1; however, version 21.1.1 is available. You should consider upgrading via the '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install upgrade pip' command. Removed build tracker: '/tmp/pipreqtrackeru1_zy58f' ERROR: invocation failed (exit code 1) python finish: installpkg /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3.zip after 675.89 seconds _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ summary __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ERROR: python: InvocationError for command /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install existsaction w v .tox/.tmp/package/1/sage_conf9.3.zip (exited with code 1) cleanup /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3.zip
comment:100 Changed 3 months ago by
 Status changed from needs_review to needs_work
No, no matter how many times I restart tox, it errors out with
make noprintdirectory singularnodeps sagelogger p 'SAGE_CHECK=no sagespkg singular4.2.0p1+20210406+sage' '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/singular4.2.0p1+20210406+sage.log' [singular4.2.0p1+20210406+sage] installing. Log file: /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/singular4.2.0p1+20210406+sage.log [palp2.11] successfully installed. make noprintdirectory ratpointsnodeps make[3]: *** No rule to make target '../pkgs/sage_conf/src/bin/sageenvconfig', needed by '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/local/var/lib/sage/installed/sage_conf9.3'. Stop. make[3]: *** Waiting for unfinished jobs.... sagelogger p 'SAGE_CHECK=no sagespkg ratpoints2.1.3.p5' '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/ratpoints2.1.3.p5.log' [ratpoints2.1.3.p5] installing. Log file: /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/logs/pkgs/ratpoints2.1.3.p5.log [ratpoints2.1.3.p5] successfully installed. [ecl21.2.1] successfully installed. [gap4.11.0.p1] successfully installed. [singular4.2.0p1+20210406+sage] successfully installed. make[2]: *** [Makefile:2150: allbuild] Error 2 make[2]: Leaving directory '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/build/make' real 9m16.619s user 23m45.919s sys 1m54.965s *************************************************************** Error building Sage. The following package(s) may have failed to build (not necessarily during this run of 'make allbuild'): It is safe to delete any log files and build directories, but they contain information that is helpful for debugging build problems. WARNING: If you now run 'make' again, the build directory of the same version of the package will, by default, be deleted. Set the environment variable SAGE_KEEP_BUILT_SPKGS=yes to prevent this. make[1]: *** [Makefile:39: allbuild] Error 1 make[1]: Leaving directory '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu' make: *** [Makefile:16: build] Error 2 Running cd /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu && (. ./.homebrewbuildenv 2> /dev/null  :) && ./configure prefix=/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/.sage/sage9.3cpython38x86_64linuxgnu/local withpython=/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python enablebuildasroot withsystempython3=force disablenotebook disablesagelib error: make buildlocal failed Building wheel for sageconf (PEP 517) ... error ERROR: Failed building wheel for sageconf Failed to build sageconf ERROR: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly Exception information: Traceback (most recent call last): File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/cli/base_command.py", line 189, in _main status = self.run(options, args) File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/cli/req_command.py", line 178, in wrapper return func(self, options, args) File "/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/lib/python3.8/sitepackages/pip/_internal/commands/install.py", line 358, in run raise InstallationError( pip._internal.exceptions.InstallationError: Could not build wheels for sageconf which use PEP 517 and cannot be installed directly WARNING: You are using pip version 21.0.1; however, version 21.1.1 is available. You should consider upgrading via the '/mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install upgrade pip' command. Removed build tracker: '/tmp/pipreqtracker9ti8dn54' ERROR: invocation failed (exit code 1) python finish: installpkg /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3.zip after 597.56 seconds _________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ summary __________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________ ERROR: python: InvocationError for command /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/python/bin/python m pip install existsaction w v .tox/.tmp/package/1/sage_conf9.3.zip (exited with code 1) cleanup /mnt/opt/Sage/sageclang/src/pkgs/sage_confpypi/.tox/.tmp/package/1/sage_conf9.3.zip
comment:101 Changed 2 months ago by
 Commit changed from df2f8d11a2d204b5e5197b968ab1bb8e62486508 to b4b3ec80489aa21324724c2c7ead38d37061fa00
Branch pushed to git repo; I updated commit sha1. New commits:
b4b3ec8  Merge tag '9.4.beta0' into t/29039/pip_installable_package_sage_bootstrap

comment:102 Changed 2 months ago by
 Commit changed from b4b3ec80489aa21324724c2c7ead38d37061fa00 to e1d2df08aa14e47a780c51c01073d58f4c7b3426
Branch pushed to git repo; I updated commit sha1. New commits:
e1d2df0  Merge tag '9.4.beta1' into t/29039/pip_installable_package_sage_bootstrap

comment:103 Changed 2 months ago by
 Dependencies #30913 deleted
comment:104 Changed 6 weeks ago by
 Commit changed from e1d2df08aa14e47a780c51c01073d58f4c7b3426 to 900e621eca31b625831b14918d1a34cf61884d07
comment:105 Changed 6 weeks ago by
 Dependencies set to #31577
comment:106 Changed 6 weeks ago by
 Commit changed from 900e621eca31b625831b14918d1a34cf61884d07 to 89bdadf349474996b7a4c786b82471273eeb1c4d
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
40abff0  pkgs/sagesetup/setup.cfg: README.rst, not README.md

782b64b  pkgs/sagemathstandard: Remove symlink to SAGE_ROOT/src/sage_setup

cf606f7  build/bin/writedockerfile.sh: ADD src/sage_setup before building TARGETS_PRE

a1b8832  build/pkgs/sage_setup/spkgsrc: Use python3

9270417  tox.ini, build/bin/writedockerfile.sh, tox*.yml: Use allsagelocal as default TARGETS_PRE

870f9aa  build/pkgs/memory_allocator/installrequires.txt: Fix filename

61c25a0  build/pkgs/sage_conf/installrequires.txt: New

8d432ca  Merge #29847

26e16c7  Add configure option disablesagelib

89bdadf  Add sage_root to sage_bootstrap

comment:107 Changed 6 weeks ago by
 Commit changed from 89bdadf349474996b7a4c786b82471273eeb1c4d to 3dc9618377133f7983a374630cd3459568a3a10a
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
3dc9618  pkgs/sageconf_pypi: New

comment:108 Changed 6 weeks ago by
 Description modified (diff)
comment:109 Changed 6 weeks ago by
 Commit changed from 3dc9618377133f7983a374630cd3459568a3a10a to e825a9f554fcc4eda8665fa9b9f708d6382b324d
comment:110 Changed 6 weeks ago by
 Dependencies changed from #31577 to #31577, #32057
comment:111 Changed 6 weeks ago by
 Commit changed from e825a9f554fcc4eda8665fa9b9f708d6382b324d to fe8019a075a5870c0ecbb032c34fc8579bdd90d9
comment:112 Changed 6 weeks ago by
 Commit changed from fe8019a075a5870c0ecbb032c34fc8579bdd90d9 to 8c0ccf319cf038e836c38f494d380f30eb5413ca
Branch pushed to git repo; I updated commit sha1. New commits:
8c0ccf3  src/pkgs/sage_conf*/README.rst: Unify the files

comment:113 Changed 6 weeks ago by
 Commit changed from 8c0ccf319cf038e836c38f494d380f30eb5413ca to 2181f10720e2c6531b33d84be0cc0aeb8e660030
Branch pushed to git repo; I updated commit sha1. New commits:
2181f10  pkgs/sageconf_pypi: Fixup

comment:114 Changed 6 weeks ago by
 Status changed from needs_work to needs_review
comment:115 Changed 6 weeks ago by
 Commit changed from 2181f10720e2c6531b33d84be0cc0aeb8e660030 to 44b1e8d88affb78b8c06f478f79d2414b4efa232
Branch pushed to git repo; I updated commit sha1. New commits:
44b1e8d  Makefile (pypisdists): Add dependency on sage_setup

comment:116 Changed 6 weeks ago by
 Dependencies changed from #31577, #32057 to #31577, #32057, #29847
comment:117 Changed 5 weeks ago by
 Dependencies changed from #31577, #32057, #29847 to #31577, #32057, #29847, #32071
comment:118 Changed 5 weeks ago by
I tried testing this over a recent pynac 0.7.28 branch, and it errors out as it apparently does not know how to pull upstream tarballs not from the mirrors. Missing enabledownloadfromupstreamurl
missing somewhere?
comment:119 Changed 5 weeks ago by
 Commit changed from 44b1e8d88affb78b8c06f478f79d2414b4efa232 to 1596cb58ab2eb42e62edabc5e94c6bf9e15a46ac
Branch pushed to git repo; I updated commit sha1. New commits:
1596cb5  pkgs/sageconf_pypi/setup.py: Use enabledownloadfromupstreamurl

comment:120 Changed 5 weeks ago by
 Status changed from needs_review to positive_review
OK, this works
comment:121 Changed 5 weeks ago by
Thank you!
comment:122 Changed 5 weeks ago by
 Commit changed from 1596cb58ab2eb42e62edabc5e94c6bf9e15a46ac to 1dd45a23d5fca5fecb4bc9fdb840e3b337e16323
 Status changed from positive_review to needs_review
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
380b42d  Update paths in comments

a7d6a3e  build/bin/writedockerfile.sh: ADD pkgs

7568dc6  Merge tag '9.4.beta4' into t/31577/move_sage_root_build_pkgs___src_to_sage_root____rename_sage_root_build_to_sage_root_sage_bootstrap

1dd45a2  Merge #31577

comment:123 Changed 5 weeks ago by
 Commit changed from 1dd45a23d5fca5fecb4bc9fdb840e3b337e16323 to 569b188b46fc42a015af7c30fdf0b3699ccd4c19
comment:124 Changed 5 weeks ago by
 Status changed from needs_review to positive_review
comment:125 Changed 11 days ago by
 Commit changed from 569b188b46fc42a015af7c30fdf0b3699ccd4c19 to cfd0c254d78f2f5fb5004e168235e5b520e43717
 Status changed from positive_review to needs_review
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
35b8b74  Merge tag '9.4.beta5' into t/29847/install_sage_setup_with_pip__move_sage_include_directories__cython_aliases_from_sage_env_to_sage_setup

773ef29  Merge #32073

9aa1f10  Merge #29585

8c5ee65  build/pkgs/pip/installrequires.txt: Update lower bound

ed5f9f0  Merge tag '9.4.beta6' into t/32046/use_pip___use_feature_in_tree_build__replace_use_of_sdh_setup_bdist_wheel_by_sdh_pip_install

6b1b222  Merge #32046

cfd0c25  Merge #29847

comment:126 Changed 11 days ago by
 Milestone changed from sage9.4 to sage9.5
New commits:
Add sage_root to sage_bootstrap