Opened 6 years ago
Closed 2 years ago
#21559 closed enhancement (fixed)
Install src/bin scripts by sagelib's setup.py, not make
Reported by:  mkoeppe  Owned by:  

Priority:  major  Milestone:  sage9.2 
Component:  build  Keywords:  
Cc:  jdemeyer, vbraun, embray, isuruf, jhpalmieri, fbissey, kcrisman, mjo, arojas, dimpase  Merged in:  
Authors:  Matthias Koeppe  Reviewers:  John Palmieri, François Bissey 
Report Upstream:  N/A  Work issues:  
Branch:  19d7f67 (Commits, GitHub, GitLab)  Commit:  19d7f677eab07dc41e3f1e71b9a37f3b58560c39 
Dependencies:  #29111  Stopgaps: 
Description (last modified by )
This ticket cleans up how the contents of src/bin
are installed. One of its goals is to make the sagelib
installation done by src/setup.py
selfcontained regarding the scripts that it needs at runtime.
Previous tickets moved some of the scripts to build/bin
(no longer installed in SAGE_LOCAL
).
In followup tickets, src/bin
could be cleaned up more.
 See the list of files added on this ticket to
src/setup.py
for beginnings of a categorization.  #29850: Install
src/bin/sageenvconfig
as part ofsage_conf
 #29847: Install
sage_setup
withsetuptools
as a prerequisite for building/installing sagelib, install sagethedistribution scripts as "scripts"
Change History (71)
comment:1 Changed 6 years ago by
 Cc embray added
 Description modified (diff)
comment:2 followup: ↓ 3 Changed 6 years ago by
comment:3 in reply to: ↑ 2 Changed 6 years ago by
comment:4 Changed 6 years ago by
 Description modified (diff)
comment:5 Changed 4 years ago by
 Description modified (diff)
 Milestone changed from sage7.5 to sage8.3
comment:6 Changed 4 years ago by
 Milestone changed from sage8.3 to sage8.4
update milestone 8.3 > 8.4
comment:7 Changed 3 years ago by
 Description modified (diff)
 Milestone changed from sage8.4 to sage8.8
comment:8 Changed 3 years ago by
 Milestone sage8.8 deleted
As the Sage8.8 release milestone is pending, we should delete the sage8.8 milestone for tickets that are not actively being worked on or that still require significant work to move forward. If you feel that this ticket should be included in the next Sage release at the soonest please set its milestone to the next release milestone (sage8.9).
comment:9 Changed 3 years ago by
 Cc isuruf added
comment:10 Changed 3 years ago by
 Cc jhpamieri fbissey added
comment:11 Changed 3 years ago by
 Cc jhpalmieri added; jhpamieri removed
comment:12 Changed 2 years ago by
 Description modified (diff)
 Milestone set to sage9.1
comment:13 Changed 2 years ago by
 Description modified (diff)
comment:14 Changed 2 years ago by
 Cc kcrisman added
 Description modified (diff)
comment:15 Changed 2 years ago by
 Branch set to u/mkoeppe/changesrcbininstallation
comment:16 Changed 2 years ago by
 Commit set to 428ea11b2a4ac1c6754f9981d0b732e593e02aac
 Description modified (diff)
 Summary changed from Move sagethedistribution scripts from src/bin to build/bin to Move sagethedistribution scripts from src/bin to build/bin; install remaining src/bin scripts by setup.py, not make
comment:17 Changed 2 years ago by
 Commit changed from 428ea11b2a4ac1c6754f9981d0b732e593e02aac to 1b3c860afdb5368545541e989975881a4c57ca1c
Branch pushed to git repo; I updated commit sha1. New commits:
1b3c860  Move src/bin/sagestarts to build/bin/

comment:18 Changed 2 years ago by
 Description modified (diff)
 Status changed from new to needs_review
comment:19 Changed 2 years ago by
 Description modified (diff)
comment:20 Changed 2 years ago by
 Commit changed from 1b3c860afdb5368545541e989975881a4c57ca1c to 546d1b07856ea236ad6a9021f21109b0082ca82d
Branch pushed to git repo; I updated commit sha1. New commits:
546d1b0  Merge tag '9.1.beta7' into t/21559/changesrcbininstallation

comment:21 Changed 2 years ago by
 Cc mjo added
comment:22 Changed 2 years ago by
This ticket could use some help from distribution packagers...
comment:23 Changed 2 years ago by
 Commit changed from 546d1b07856ea236ad6a9021f21109b0082ca82d to dddd4628522f62c7f78315b507116117a580912d
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
2850b50  Move src/bin/testc*.sh to build/bin/, delete src/bin/sageREADMEosx.txt

4466fd6  Install src/bin/* by setup.py [scripts] instead of by build/make/deps

26b7596  Move src/bin/sagestarts to build/bin/

dddd462  Remove obsolete script sagepkg (creates oldstyle spkg archives)

comment:24 Changed 2 years ago by
 Commit changed from dddd4628522f62c7f78315b507116117a580912d to 9825799e82c8201359faf2e7db103b9f52ba2aef
Branch pushed to git repo; I updated commit sha1. New commits:
9825799  src/bin/sageunzip: Remove (unused script)

comment:25 Changed 2 years ago by
 Description modified (diff)
comment:26 Changed 2 years ago by
 Status changed from needs_review to needs_work
comment:27 Changed 2 years ago by
 Dependencies set to #23769, #29381, #29383
comment:28 Changed 2 years ago by
 Description modified (diff)
comment:29 Changed 2 years ago by
 Description modified (diff)
comment:30 Changed 2 years ago by
 Cc arojas added
 Description modified (diff)
comment:31 Changed 2 years ago by
This breaks if you try to install sagelib with setup.py without running configure first (as some distros do) since sageenvconfig doesn't exist in that case. At least sageenv and sageenvconfig should be optional.
comment:32 followup: ↓ 33 Changed 2 years ago by
Can this build also configure a better link or communication between SAGE_LOCAL or SAGE_SCRIPTS_DIR variables?
comment:33 in reply to: ↑ 32 Changed 2 years ago by
Replying to ghShlokatadistance:
Can this build also configure a better link or communication between SAGE_LOCAL or SAGE_SCRIPTS_DIR variables?
Take a look at #25486
comment:34 Changed 2 years ago by
 Milestone changed from sage9.1 to sage9.2
comment:35 Changed 2 years ago by
 Dependencies changed from #23769, #29381, #29383 to #29702
comment:36 Changed 2 years ago by
 Commit changed from 9825799e82c8201359faf2e7db103b9f52ba2aef to 7c8a0304aff5c2f6376fa7c4b4acf56d35766f31
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
5db5318  Trac #29345: remove "break" statements from AC_SEARCH_LIBS.

e810ad1  Trac #29345: don't use sage's config.status for the lrcalc build.

93c9921  Trac #29345: replace the function that populates the CVXOPT_* variables.

0e66a0a  Trac #29345: add Dima's SPKG patches for ksh compatibility.

df3f05e  build/make/Makefile.in [SCRIPT_PACKAGE_templ]: cd into the SPKG directory; adjust spkginstall scripts

5372065  Merge branch 't/29793/script_packages_should_cd_into_the_spkg_directory' into t/29411/make_sagelib_a_script_package

c166b97  Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup

cc30471  build/bin/writedockerfile.sh: Do not ADD removed file src/Makefile.in

8a41326  Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup

7c8a030  Install src/bin/* by setup.py [scripts] instead of by build/make/Makefile.in

comment:37 Changed 2 years ago by
 Description modified (diff)
 Summary changed from Move sagethedistribution scripts from src/bin to build/bin; install remaining src/bin scripts by setup.py, not make to Install src/bin scripts by sagelib's setup.py, not make
comment:38 Changed 2 years ago by
 Commit changed from 7c8a0304aff5c2f6376fa7c4b4acf56d35766f31 to 64ec6ae70a8d10ebec7158eecebff996592d273b
Branch pushed to git repo; I updated commit sha1. New commits:
64ec6ae  src/setup.py [scripts]: Categorize scripts

comment:39 Changed 2 years ago by
 Status changed from needs_work to needs_review
comment:40 Changed 2 years ago by
 Commit changed from 64ec6ae70a8d10ebec7158eecebff996592d273b to 1ee054c0da2494f6bdbe5d4bd6088e0d39c0d3f8
Branch pushed to git repo; I updated commit sha1. New commits:
1ee054c  Fixup

comment:41 Changed 2 years ago by
 Commit changed from 1ee054c0da2494f6bdbe5d4bd6088e0d39c0d3f8 to daa045c01c6ae4767213abf9dcc1820829200999
Branch pushed to git repo; I updated commit sha1. New commits:
daa045c  src/setup.py: Do not install deprecated script bin/sagefixpkgchecksums

comment:42 Changed 2 years ago by
(this script is deleted in #29111)
comment:43 Changed 2 years ago by
 Cc dimpase added
comment:44 Changed 2 years ago by
 Dependencies changed from #29702 to #29702, #29111, #29920
comment:45 Changed 2 years ago by
 Commit changed from daa045c01c6ae4767213abf9dcc1820829200999 to e3d9cf23fcce51f679878b86ce462d91d3c41ea5
Branch pushed to git repo; I updated commit sha1. New commits:
e3d9cf2  src/setup.py: Do not install deprecated scripts sagelist{experimental,optional,standard }

comment:46 Changed 2 years ago by
(these scripts are deleted in #29920)
comment:47 followup: ↓ 48 Changed 2 years ago by
A bit much to ask but is there a way to override the scripts
variable in setup.py
? Apart from patching that is. I and other distros will pick and choose what we install and certainly, even where. We can do things after install is run if needs be.
But building and providing our own list would be a nice to have.
comment:48 in reply to: ↑ 47 Changed 2 years ago by
Replying to fbissey:
A bit much to ask but is there a way to override the
scripts
variable insetup.py
? Apart from patching that is. I and other distros will pick and choose what we install and certainly, even where. We can do things after install is run if needs be.But building and providing our own list would be a nice to have.
Adding hooks for distributors is not a good direction for development. It's better to sort out in sagelib itself which scripts should be considered part of sagelib, and which ones are provided by sagethedistribution instead.
comment:49 followups: ↓ 50 ↓ 52 Changed 2 years ago by
This can be done on followup tickets of this ticket. You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".
comment:50 in reply to: ↑ 49 Changed 2 years ago by
Replying to mkoeppe:
This can be done on followup tickets of this ticket. You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".
I'll think about it. Like I said it is a "nice to have", not a must have. But this will mark a transition for distros. Before this, most of us were manually installing a selection of scripts. Suddenly the install won't be manual anymore :)
comment:51 Changed 2 years ago by
In fact, the scripts that belong to sagethedistribution could perhaps be assigned to sage_setup
(#29847)  this distutils package that would be installed by sagethedistribution but not by distros.
comment:52 in reply to: ↑ 49 ; followup: ↓ 53 Changed 2 years ago by
Replying to mkoeppe:
This can be done on followup tickets of this ticket. You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".
Here are some thoughts:
Perhaps could just be deleted:
sagecython
: #29923sagenativeexecute
: look at the script and see #9386.sageopen
: I wonder if the comment in that script is still valid; perhaps it can be deleted, or done purely in Python in sage/misc/viewer.py, the only place it's used. Maybe I'll look into this.sagepypkglocation
: apparently unusedsageunzip
is in the listed, but the script itself is gone, isn't it?
Doctestingrelated scripts. Where do these go?
 sageruntests
 coverage scripts
 sagefixdoctests
 sagestartuptime
In the libexec group:
 sagecleaner?
Only relevant for sagethedistribution:
sagedownload
sageclonesource
sagesdist
sagelocation
comment:53 in reply to: ↑ 52 Changed 2 years ago by
Replying to jhpalmieri:
Doctestingrelated scripts. Where do these go?
 sageruntests
 coverage scripts
 sagefixdoctests
Per #29111, they are sagelib scripts.
 sagestartuptime
Not sure about this one.
comment:54 Changed 2 years ago by
 Commit changed from e3d9cf23fcce51f679878b86ce462d91d3c41ea5 to 455c294091537ca92df0bfbf39fae38816c808bf
Branch pushed to git repo; I updated commit sha1. New commits:
455c294  src/setup.py: Do not install removed script bin/sageunzip

comment:55 Changed 2 years ago by
 Commit changed from 455c294091537ca92df0bfbf39fae38816c808bf to b214410e12972a06e4cab87b3225d3d8aac4db7b
Branch pushed to git repo; I updated commit sha1. New commits:
b214410  Classify doctestingrelated scripts

comment:56 Changed 2 years ago by
 Commit changed from b214410e12972a06e4cab87b3225d3d8aac4db7b to 8d10bb99f339795601993cb8b2899f544471fd68
Branch pushed to git repo; I updated commit sha1. New commits:
8d10bb9  Categorize some uncategorized scripts

comment:57 Changed 2 years ago by
 Commit changed from 8d10bb99f339795601993cb8b2899f544471fd68 to 6b04075aca84249a2418a365f31ea4a0978461a8
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
05ba86a  Merge branch 't/29884/src_doc_bootstrap__generate_src_doc_en_reference_repl_options_rst' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution

72c7e23  src/doc/en/reference/repl/options.rst: Replace copypasta by include of generated file options.txt

e9a883d  trac 29111: revising "sage advanced" message.

5dca421  trac 29111: more reorganization of "sage advanced" message

8f0ee05  trac 29111: delete sagefixpkgchecksums

21fc231  trac 29111: re "sage advanced" message:

875940d  trac 29111: more tinkering. Fix doctests in tests/cmdline.py.

4a3d36e  Move 'sage app' back to src/bin/sage

3a0193c  src/bin/sage: Remove handling of 'sage axiom'

6b04075  Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/changesrcbininstallation

comment:58 Changed 2 years ago by
 Commit changed from 6b04075aca84249a2418a365f31ea4a0978461a8 to 831cc09d32828144a27f7bc8c8b2f70150ed0047
Branch pushed to git repo; I updated commit sha1. New commits:
9c7116b  src/bin/sagelistoptional, sagelistexperimental, sageliststandard: Remove deprecated scripts

831cc09  Merge branch 't/29920/remove_deprecated_scripts_sage_list_optional__sage_list_experimental__sage_list_standard' into t/21559/changesrcbininstallation

comment:59 Changed 2 years ago by
 Commit changed from 831cc09d32828144a27f7bc8c8b2f70150ed0047 to e3eca857db71b602cf2c7c16755ab2cdeaca1773
Branch pushed to git repo; I updated commit sha1. New commits:
a56dc35  Merge tag '9.2.beta1' into t/29702/public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup

e3eca85  Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/21559/changesrcbininstallation

comment:60 Changed 2 years ago by
I run into problems unless I make this change:

src/setup.py
diff git a/src/setup.py b/src/setup.py index 9970525cf9..85a32a5962 100755
a b code = setup(name = 'sage', 162 162 'bin/sagerebaseall.sh', 163 163 'bin/sagerst2sws', 164 164 'bin/sagerst2txt', 165 'bin/sagersyncdist',166 165 'bin/sagerun', 167 166 'bin/sageruncython', 168 167 'bin/sagestartuptime.py',
comment:61 Changed 2 years ago by
 Commit changed from e3eca857db71b602cf2c7c16755ab2cdeaca1773 to 7d29141662fe9ffd80c61e8867153244004d0a37
Branch pushed to git repo; I updated commit sha1. New commits:
7d29141  src/setup.py: Do not install removed script sagersyncdist

comment:62 Changed 2 years ago by
 Description modified (diff)
comment:63 Changed 2 years ago by
Needs review
comment:64 Changed 2 years ago by
 Reviewers set to John Palmieri, François Bissey
 Status changed from needs_review to positive_review
Looks okay to me.
comment:65 Changed 2 years ago by
Thanks!
comment:66 Changed 2 years ago by
 Commit changed from 7d29141662fe9ffd80c61e8867153244004d0a37 to c5f7f2fb29d376c6bb103e2f070f92690c689161
 Status changed from positive_review to needs_review
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
c5f7f2f  Merge tag '9.2.beta3' into t/21559/changesrcbininstallation

comment:67 Changed 2 years ago by
 Status changed from needs_review to positive_review
comment:68 Changed 2 years ago by
 Commit changed from c5f7f2fb29d376c6bb103e2f070f92690c689161 to 19d7f677eab07dc41e3f1e71b9a37f3b58560c39
 Status changed from positive_review to needs_review
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
3953671  Merge tag '9.2.beta4' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution

19d7f67  Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/changesrcbininstallation

comment:69 Changed 2 years ago by
 Dependencies changed from #29702, #29111, #29920 to #29111
comment:70 Changed 2 years ago by
 Status changed from needs_review to positive_review
comment:71 Changed 2 years ago by
 Branch changed from u/mkoeppe/changesrcbininstallation to 19d7f677eab07dc41e3f1e71b9a37f3b58560c39
 Resolution set to fixed
 Status changed from positive_review to closed
+1
Also
sagepipinstall
of #21441, but I might wait until that ticket is merged first, then move that script along with the others.