Opened 2 years ago
Closed 21 months ago
#29950 closed enhancement (fixed)
Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add spkgsrc and tox.ini
Reported by:  mkoeppe  Owned by:  

Priority:  major  Milestone:  sage9.2 
Component:  refactoring  Keywords:  
Cc:  fbissey, jhpalmieri, dimpase, ghkliem, isuruf, vbraun  Merged in:  
Authors:  Matthias Koeppe  Reviewers:  Dima Pasechnik 
Report Upstream:  N/A  Work issues:  
Branch:  ccc67b0 (Commits, GitHub, GitLab)  Commit:  ccc67b0356373330164dd75060f574c184142473 
Dependencies:  #29701  Stopgaps: 
Description (last modified by )
As a followup to #29411, we set up a separate directory build/pkgs/sagelib/src
with symlinks into SAGE_ROOT/src
so that sagelib's setup.py
does not have access to the whole tree SAGE_ROOT/src
.
We fix setup.py sdist
, which can be tested using spkgsrc
(which builds a pipinstallable tarball in upstream/
) and tox
:
./sage sh c '(cd build/pkgs/sagelib/src && tox v v)' ./sage sh c 'build/pkgs/sagelib/src/.tox/python/bin/sageipython'
The call of ./sage sh
is necessary so that SAGE_LOCAL
is set.
Moreover, for scripts that use sageenv
, also SAGE_ENV_SOURCED
is crucial for the correct operation; otherwise sageenv
sets things based on SAGE_LOCAL
again.
Change History (44)
comment:1 Changed 23 months ago by
 Branch set to u/mkoeppe/build_sagelib_using_installed_sage_setup
comment:2 Changed 23 months ago by
 Commit set to 1ab75a99c46a00e7424a5d91366097fc32c9a13a
comment:3 Changed 23 months ago by
 Summary changed from Build sagelib using the installed sage_setup to Build sagelib using the installed sage_setup, add spkgsrc
comment:4 Changed 23 months ago by
 Work issues set to add spkgsrc
comment:5 Changed 23 months ago by
comment:6 Changed 23 months ago by
 Commit changed from 1ab75a99c46a00e7424a5d91366097fc32c9a13a to a29cd2d79dd84d9653f9f0ea38466f846edb0e4f
comment:7 Changed 23 months ago by
 Work issues changed from add spkgsrc to Fixup MANIFEST.in
comment:8 Changed 23 months ago by
sdist is broken: Files named **
end up in the tar file.
comment:9 Changed 23 months ago by
 Dependencies changed from #29411, #29847 to #29411, #29847, #29701
comment:11 Changed 23 months ago by
 Commit changed from a29cd2d79dd84d9653f9f0ea38466f846edb0e4f to c3b8f8f057647eca8725a7a35a415c1cc3e19d4b
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
7f8850a  src/module_list.py: Document that it is obsolete

26a85a4  Merge branch 't/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' into t/29701/replace_use_of_module_list_optionalextension

5d5803e  src/sage/graphs/planarity.pyx: Add forgotten distutils directive

a0be9b6  Merge branch 't/29790/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_5__sage_graphs_' into t/29701/replace_use_of_module_list_optionalextension

14396af  Merge tag '9.2.beta2' into t/29701/replace_use_of_module_list_optionalextension

821e337  Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup

eeb6ba4  build/pkgs/sagelib/src/requirements.txt, tox.ini: New

ce8b84d  build/pkgs/sagelib/src/MANIFEST.in: Prune sage/ext/interpreters

a9a1335  build/pkgs/sagelib/src/requirements.txt: Add packages required for runtime

c3b8f8f  Suggest tox v v

comment:12 Changed 23 months ago by
 Description modified (diff)
 Summary changed from Build sagelib using the installed sage_setup, add spkgsrc to Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add spkgsrc and tox.ini
 Work issues changed from Fixup MANIFEST.in to Rebase away from #29847
comment:13 Changed 23 months ago by
 Commit changed from c3b8f8f057647eca8725a7a35a415c1cc3e19d4b to 9955a3ff69870ff844e5ebb45cf47c4b6f214263
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
cdf501f  build/pkgs/sagelib/spkgsrc: New

ef4f5b8  Move src/MANIFEST.in to build/pkgs/sagelib/MANIFEST.in, fix up

233f6ba  build/pkgs/sagelib/src/requirements.txt, tox.ini: New

177cd4c  build/pkgs/sagelib/src/MANIFEST.in: Prune sage/ext/interpreters

07b29fb  build/pkgs/sagelib/src/requirements.txt: Add packages required for runtime

ad33705  Suggest tox v v

481e212  .gitignore: Ignore generated files in build/pkgs/*/src

9955a3f  Build sagelib from build/pkgs/sagelib/src

comment:14 Changed 23 months ago by
 Dependencies changed from #29411, #29847, #29701 to #29411, #29701
 Work issues Rebase away from #29847 deleted
comment:15 Changed 23 months ago by
from .maxima import maxima, Maxima File "/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/build/pkgs/sagelib/src/.tox/python/lib/python3.7/sitepackages/sage/interfaces/maxima.py", line 1236, in <module> script_subdirectory=None) File "/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/build/pkgs/sagelib/src/.tox/python/lib/python3.7/sitepackages/sage/interfaces/maxima.py", line 528, in __init__ raise RuntimeError('You must get the file local/bin/sagemaxima.lisp') RuntimeError: You must get the file local/bin/sagemaxima.lisp
comment:16 Changed 23 months ago by
 Dependencies changed from #29411, #29701 to #29411, #29701, #27171
comment:17 Changed 23 months ago by
 Dependencies changed from #29411, #29701, #27171 to #29411, #29701, #21559
comment:18 Changed 23 months ago by
 Commit changed from 9955a3ff69870ff844e5ebb45cf47c4b6f214263 to 08f5175bdf71236ccf3e80f6efaffa83d5edc532
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
875940d  trac 29111: more tinkering. Fix doctests in tests/cmdline.py.

4a3d36e  Move 'sage app' back to src/bin/sage

3a0193c  src/bin/sage: Remove handling of 'sage axiom'

6b04075  Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/changesrcbininstallation

9c7116b  src/bin/sagelistoptional, sagelistexperimental, sageliststandard: Remove deprecated scripts

831cc09  Merge branch 't/29920/remove_deprecated_scripts_sage_list_optional__sage_list_experimental__sage_list_standard' into t/21559/changesrcbininstallation

a56dc35  Merge tag '9.2.beta1' into t/29702/public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup

e3eca85  Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/21559/changesrcbininstallation

7d29141  src/setup.py: Do not install removed script sagersyncdist

08f5175  Merge branch 'u/mkoeppe/changesrcbininstallation' of git://trac.sagemath.org/sage into t/29950/build_sagelib_using_installed_sage_setup

comment:19 Changed 23 months ago by
 Description modified (diff)
comment:20 Changed 23 months ago by
./sage sh c '(cd build/pkgs/sagelib/src && tox r v v)' ... RuntimeError: libSingular not founda working Singular install in $SAGE_LOCAL is required for Sage to work ERROR: InvocationError for command /Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/build/pkgs/sagelib/src/.tox/python/bin/python c 'import sys; "" in sys.path and sys.path.remove(""); import sage.all' (exited with code 1)
This is because of:
$ build/pkgs/sagelib/src/.tox/python/bin/python Python 3.7.7 (v3.7.7:d7c567b08f, Mar 10 2020, 02:56:16) [Clang 6.0 (clang600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import sage.env >>> sage.env.SINGULAR_SO >>> sage.env.GAP_SO >>> sage.env.SAGE_LOCAL '/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/build/pkgs/sagelib/src/.tox/python' >>>
comment:21 Changed 23 months ago by
Passing the correct SAGE_LOCAL
suffices to fix this, of course. But the singular library location should be configured separately (#29024).
comment:22 Changed 23 months ago by
 Commit changed from 08f5175bdf71236ccf3e80f6efaffa83d5edc532 to 516358b66059dbf458ed7c970ff05923b539b71c
Branch pushed to git repo; I updated commit sha1. New commits:
516358b  build/pkgs/sagelib/src/tox.ini: Pass SAGE_LOCAL

comment:23 Changed 23 months ago by
 Commit changed from 516358b66059dbf458ed7c970ff05923b539b71c to eaced098339d5c7655244667b3dd1c90f5fd7f9e
Branch pushed to git repo; I updated commit sha1. New commits:
d62da15  sage_setup.command.sage_build: Add the extensions to the distribution

5e577ab  Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup

eaced09  build/pkgs/sagelib: Add remaining Python dependencies for sage.all

comment:24 Changed 23 months ago by
 Description modified (diff)
comment:25 Changed 23 months ago by
 Description modified (diff)
 Status changed from new to needs_review
comment:26 Changed 23 months ago by
 Description modified (diff)
comment:27 Changed 23 months ago by
The sage
script relies on PATH
to be set correctly when invoking auxiliary scripts such as sageeval
. Hence the following does not work correctly:
$ ./sage sh c 'build/pkgs/sagelib/src/.tox/python/bin/sage c "print(sys.path)"' ['/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/src/bin', '/Library/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/libdynload', '/Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/local/lib/python3.7/sitepackages']
comment:28 Changed 23 months ago by
This will be fixed by #30013 (src/bin/sageenv
: Make sure SAGE_SCRIPTS_DIR
is at the beginning of the PATH
).
comment:29 Changed 23 months ago by
 Commit changed from eaced098339d5c7655244667b3dd1c90f5fd7f9e to b8e366ec56a9d3527c81df9d8d4206afc2562be4
Branch pushed to git repo; I updated commit sha1. New commits:
b8e366e  build/pkgs/sagelib/spkgsrc: chmod +x

comment:30 Changed 23 months ago by
 Cc jhpalmieri dimpase ghkliem isuruf added
comment:31 Changed 23 months ago by
All dependencies of this ticket are green. Ready for review.
comment:32 Changed 23 months ago by
 Commit changed from b8e366ec56a9d3527c81df9d8d4206afc2562be4 to 72443711047daa51d1ba92b087ec9022b2eeef06
Branch pushed to git repo; I updated commit sha1. New commits:
c5f7f2f  Merge tag '9.2.beta3' into t/21559/changesrcbininstallation

0473ef3  Merge branch 't/21559/changesrcbininstallation' into t/29950/build_sagelib_using_installed_sage_setup

06a3609  sage_setup.find.find_python_sources: Add benchmark doctest

174626c  Merge branch 't/29786/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_4__sage_rings_' into t/29701/replace_use_of_module_list_optionalextension

034a7f7  Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/29701/replace_use_of_module_list_optionalextension

d3c608b  Merge tag '9.2.beta3' into t/29701/replace_use_of_module_list_optionalextension

7244371  Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup

comment:33 Changed 23 months ago by
comment:34 Changed 23 months ago by
 Commit changed from 72443711047daa51d1ba92b087ec9022b2eeef06 to 4344f8940fc44d959da5d20dbea43b64181b961f
Branch pushed to git repo; I updated commit sha1. New commits:
3953671  Merge tag '9.2.beta4' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution

19d7f67  Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/changesrcbininstallation

4344f89  Merge branch 't/21559/changesrcbininstallation' into t/29950/build_sagelib_using_installed_sage_setup

comment:35 Changed 22 months ago by
 Commit changed from 4344f8940fc44d959da5d20dbea43b64181b961f to 01b96b0f44f4ff7407fd2f3ce83de1cc74259d58
Branch pushed to git repo; I updated commit sha1. New commits:
fcad518  Merge tag '9.2.beta7' into t/29701/replace_use_of_module_list_optionalextension

55c3fbc  src/sage_setup/clean.py: Fix doctest

01b96b0  Merge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29950/build_sagelib_using_installed_sage_setup

comment:36 Changed 22 months ago by
 Dependencies changed from #29411, #29701, #21559 to #29701
comment:37 Changed 22 months ago by
 Reviewers set to Dima Pasechnik
 Status changed from needs_review to positive_review
lgtm
comment:38 Changed 22 months ago by
 Cc vbraun added
 Status changed from positive_review to needs_work
Failing doctest needs work as noted in #29701.
comment:39 Changed 22 months ago by
 Commit changed from 01b96b0f44f4ff7407fd2f3ce83de1cc74259d58 to 8a19fe2a9bf4ed64a48ac3110cccbbdff97684ca
Branch pushed to git repo; I updated commit sha1. New commits:
8a19fe2  build/make/Makefile.in (sagelibclean): Clean the new build location

comment:40 Changed 22 months ago by
 Commit changed from 8a19fe2a9bf4ed64a48ac3110cccbbdff97684ca to ccc67b0356373330164dd75060f574c184142473
Branch pushed to git repo; I updated commit sha1. New commits:
ccc67b0  src/sage_setup: Update cythonized_dir in doctests

comment:41 Changed 22 months ago by
 Status changed from needs_work to needs_review
comment:43 Changed 22 months ago by
Thanks!
comment:44 Changed 21 months ago by
 Branch changed from u/mkoeppe/build_sagelib_using_installed_sage_setup to ccc67b0356373330164dd75060f574c184142473
 Resolution set to fixed
 Status changed from positive_review to closed
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
.gitignore: Ignore generated files in build/pkgs/*/src
Build sagelib using the installed sage_setup