Install src/bin scripts by sagelib's setup.py, not make

[mkoeppe]

This ticket cleans up how the contents of src/bin are installed. One of its goals is to make the sagelib installation done by src/setup.py self-contained regarding the scripts that it needs at runtime.

Previous tickets moved some of the scripts to build/bin (no longer installed in SAGE_LOCAL).

In follow-up tickets, src/bin could be cleaned up more.

• See the list of files added on this ticket to src/setup.py for beginnings of a categorization.
• #29850: Install src/bin/sage-env-config as part of sage_conf
• #29847: Install sage_setup with setuptools as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"

[embray]

+1

Also sage-pip-install of #21441, but I might wait until that ticket is merged first, then move that script along with the others.

[jdemeyer]

Replying to embray:

+1

Also sage-pip-install of #21441

I already moved that in #21441 (which still needs review by the way).

[vdelecroix]

update milestone 8.3 -> 8.4

[embray]

As the Sage-8.8 release milestone is pending, we should delete the sage-8.8 milestone for tickets that are not actively being worked on or that still require significant work to move forward. If you feel that this ticket should be included in the next Sage release at the soonest please set its milestone to the next release milestone (sage-8.9).

[mkoeppe]

New commits:

​1a54a80	Move src/bin/testc*.sh to build/bin/, delete src/bin/sage-README-osx.txt
​428ea11	Install src/bin/* by setup.py [scripts] instead of by build/make/deps

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1b3c860

Move src/bin/sage-starts to build/bin/

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​546d1b0

Merge tag '9.1.beta7' into t/21559/change-src-bin-installation

[mkoeppe]

This ticket could use some help from distribution packagers...

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​2850b50	Move src/bin/testc*.sh to build/bin/, delete src/bin/sage-README-osx.txt
​4466fd6	Install src/bin/* by setup.py [scripts] instead of by build/make/deps
​26b7596	Move src/bin/sage-starts to build/bin/
​dddd462	Remove obsolete script sage-pkg (creates old-style spkg archives)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9825799

src/bin/sage-unzip: Remove (unused script)

[arojas]

This breaks if you try to install sagelib with setup.py without running configure first (as some distros do) since sage-env-config doesn't exist in that case. At least sage-env and sage-env-config should be optional.

[gh-Shlokatadistance]

Can this build also configure a better link or communication between SAGE_LOCAL or SAGE_SCRIPTS_DIR variables?

[mkoeppe]

Replying to gh-Shlokatadistance:

Can this build also configure a better link or communication between SAGE_LOCAL or SAGE_SCRIPTS_DIR variables?

Take a look at #25486

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

​5db5318	Trac #29345: remove "break" statements from AC_SEARCH_LIBS.
​e810ad1	Trac #29345: don't use sage's config.status for the lrcalc build.
​93c9921	Trac #29345: replace the function that populates the CVXOPT_* variables.
​0e66a0a	Trac #29345: add Dima's SPKG patches for ksh compatibility.
​df3f05e	build/make/Makefile.in [SCRIPT_PACKAGE_templ]: cd into the SPKG directory; adjust spkg-install scripts
​5372065	Merge branch 't/29793/script_packages_should_cd_into_the_spkg_directory' into t/29411/make_sagelib_a_script_package
​c166b97	Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup
​cc30471	build/bin/write-dockerfile.sh: Do not ADD removed file src/Makefile.in
​8a41326	Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup
​7c8a030	Install src/bin/* by setup.py [scripts] instead of by build/make/Makefile.in

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​64ec6ae

src/setup.py [scripts]: Categorize scripts

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1ee054c

Fixup

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​daa045c

src/setup.py: Do not install deprecated script bin/sage-fix-pkg-checksums

[mkoeppe]

(this script is deleted in #29111)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e3d9cf2

src/setup.py: Do not install deprecated scripts sage-list-{experimental,optional,standard}

[mkoeppe]

(these scripts are deleted in #29920)

[fbissey]

A bit much to ask but is there a way to override the scripts variable in setup.py? Apart from patching that is. I and other distros will pick and choose what we install and certainly, even where. We can do things after install is run if needs be.

But building and providing our own list would be a nice to have.

[mkoeppe]

Replying to fbissey:

A bit much to ask but is there a way to override the scripts variable in setup.py? Apart from patching that is. I and other distros will pick and choose what we install and certainly, even where. We can do things after install is run if needs be.

But building and providing our own list would be a nice to have.

Adding hooks for distributors is not a good direction for development. It's better to sort out in sagelib itself which scripts should be considered part of sagelib, and which ones are provided by sage-the-distribution instead.

[mkoeppe]

This can be done on follow-up tickets of this ticket. You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".

[fbissey]

Replying to mkoeppe:

This can be done on follow-up tickets of this ticket. You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".

I'll think about it. Like I said it is a "nice to have", not a must have. But this will mark a transition for distros. Before this, most of us were manually installing a selection of scripts. Suddenly the install won't be manual anymore :)

[mkoeppe]

In fact, the scripts that belong to sage-the-distribution could perhaps be assigned to sage_setup (#29847) -- this distutils package that would be installed by sage-the-distribution but not by distros.

[jhpalmieri]

Replying to mkoeppe:

This can be done on follow-up tickets of this ticket. You can help by categorizing the scripts in the section "Uncategorized scripts in alphabetical order".

Here are some thoughts:

Perhaps could just be deleted:

• sage-cython: #29923
• sage-native-execute: look at the script and see #9386.
• sage-open: I wonder if the comment in that script is still valid; perhaps it can be deleted, or done purely in Python in sage/misc/viewer.py, the only place it's used. Maybe I'll look into this.
• sage-pypkg-location: apparently unused
• sage-unzip is in the listed, but the script itself is gone, isn't it?

Doctesting-related scripts. Where do these go?

• sage-runtests
• coverage scripts
• sage-fix-doctests
• sage-startuptime

In the libexec group:

• sage-cleaner?

Only relevant for sage-the-distribution:

• sage-download
• sage-clone-source
• sage-sdist
• sage-location

[mkoeppe]

Replying to jhpalmieri:

Doctesting-related scripts. Where do these go?

• sage-runtests
• coverage scripts
• sage-fix-doctests

Per #29111, they are sagelib scripts.

• sage-startuptime

Not sure about this one.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​455c294

src/setup.py: Do not install removed script bin/sage-unzip

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b214410

Classify doctesting-related scripts

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​8d10bb9

Categorize some uncategorized scripts

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​05ba86a	Merge branch 't/29884/src_doc_bootstrap__generate_src_doc_en_reference_repl_options_rst' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution
​72c7e23	src/doc/en/reference/repl/options.rst: Replace copypasta by include of generated file options.txt
​e9a883d	trac 29111: revising "sage --advanced" message.
​5dca421	trac 29111: more reorganization of "sage --advanced" message
​8f0ee05	trac 29111: delete sage-fix-pkg-checksums
​21fc231	trac 29111: re "sage --advanced" message:
​875940d	trac 29111: more tinkering. Fix doctests in tests/cmdline.py.
​4a3d36e	Move 'sage -app' back to src/bin/sage
​3a0193c	src/bin/sage: Remove handling of 'sage -axiom'
​6b04075	Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/change-src-bin-installation

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9c7116b	src/bin/sage-list-optional, sage-list-experimental, sage-list-standard: Remove deprecated scripts
​831cc09	Merge branch 't/29920/remove_deprecated_scripts_sage_list_optional__sage_list_experimental__sage_list_standard' into t/21559/change-src-bin-installation

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a56dc35	Merge tag '9.2.beta1' into t/29702/public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup
​e3eca85	Merge branch 'public/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' of git://trac.sagemath.org/sage into t/21559/change-src-bin-installation

[jhpalmieri]

I run into problems unless I make this change:

src/setup.py

@@ -162 +162 @@
                  'bin/sage-rebaseall.sh',
                  'bin/sage-rst2sws',
                  'bin/sage-rst2txt',
-                 'bin/sage-rsyncdist',
                  'bin/sage-run',
                  'bin/sage-run-cython',
                  'bin/sage-startuptime.py',

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7d29141

src/setup.py: Do not install removed script sage-rsyncdist

[mkoeppe]

Needs review

[jhpalmieri]

Looks okay to me.

[mkoeppe]

Thanks!

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

​c5f7f2f

Merge tag '9.2.beta3' into t/21559/change-src-bin-installation

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

​3953671	Merge tag '9.2.beta4' into t/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution
​19d7f67	Merge branch 't/29111/specify_a_subset_of_sage_command_line_options_that_are_supported_by_sagelib___rather_than_sage_the_distribution' into t/21559/change-src-bin-installation