Improve build/pkgs/*/SPKG.rst

[Matthias Köppe]

With #29633, we now generate a full list of packages that Sage uses (with links to documentation - see #27495) for the reference manual from information in build/pkgs/*/SPKG.rst (which used to be SPKG.txt).

The purpose of this ticket is to improve the SPKG.rst files and the scripts that transform them to generate a part of the Sage reference manual.

1. If other sources have better information than what is in build/pkgs, update build/pkgs accordingly. (See ​https://wiki.sagemath.org/SageWiki)

• (done:) ​https://github.com/sagemath/website/blob/master/conf/packages.yaml (which is used for generating ​http://www.sagemath.org/links-components.html)
• (done:) ​https://wiki.sagemath.org/spkg (another incomplete list of packages)
• src/doc/en/installation/standard_packages.rst (yet another incomplete list of packages)

2. Delete/archive the obsolete other sources.

• ​https://github.com/sagemath/website/pull/212 removes packages.yaml
• ​https://wiki.sagemath.org/spkg has been cleared

3. Change the titles of the SPKG.rst files to use the format "spkg_name: Short description". This will improve the table of contents shown in ​https://doc.sagemath.org/html/en/reference/spkg/index.html, which the Sage website is pointing to after ​https://github.com/sagemath/website/pull/212

4. Extend build/bin/sage-spkg-info and use it to format additional information from build/pkgs/SPKG/.

5. In addition to the full alphabetical list of packages, also show the standard packages, the optional packages, and the experimental packages separately.

Follow-up:

• #31136 Rename SPKGs to match their PyPI name

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​7ad2d92

build/pkgs/a*/SPKG.rst: Update from https://wiki.sagemath.org/spkg

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ea524ae	build/pkgs/beautifulsoup4/SPKG.rst: New, from pypi and https://wiki.sagemath.org/spkg
​b6f40ac	build/pkgs/biopython/SPKG.rst: New, from pypi and https://wiki.sagemath.org/spkg
​316a7d2	build/pkgs/b*/SPKG.rst: Update from https://wiki.sagemath.org/spkg

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​546e15a

build/pkgs/*/SPKG.rst: Update remaining packages from https://wiki.sagemath.org/spkg

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e08928d

build/pkgs/[e-z]*/SPKG.rst: Update from website conf/packages.yaml

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​23c4903	src/doc/bootstrap: sort the SPKG list
​b98bcbb	for f in build/pkgs/*/SPKG.txt; do pandoc --from=mediawiki --to=rst $f --output=$(dirname $f)/SPKG.rst; git rm -f $f; git add $(dirname $f)/SPKG.rst; done
​047ffcb	build/pkgs/*/SPKG.rst: Remove remaining changelogs
​224059a	Fix up RST syntax
​5fe3989	Merge branch 't/30955/change_remaining_spkg_txt_to_spkg_rst' into t/29655/improve_build_pkgs___spkg_rst
​5e61388	build/pkgs/[0-b]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ba4ac57

build/pkgs/[b-c]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​897d8bd

build/pkgs/[c-d]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3775de3

build/pkgs/[e-g]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a08dad2

build/pkgs/[h-m]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'

[Matthias Köppe]

Help with the remaining packages is certainly welcome

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​5a27857	build/bin/sage-print-system-package-command: Add nix
​4c1193d	build/bin/sage-spkg: Remove redundant output, remove error message for 'sage -info' for non-normal packages
​82b95c6	build/bin/sage-spkg-info: Format more info, use RST style, use it in src/doc/bootstrap

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b3b0041

build/bin/sage-spkg-info: For Python packages, point to #29023, not #27330

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c0ccb9f	src/doc/bootstrap: Generate packages by type
​6463353	build/pkgs/[m-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- for all remaining packages from packages.yaml

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f5ac0e0	build/pkgs/[n-s]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'
​109b3c2	sage --package create --pypi: Adjust to new format of the SPKG.rst title
​368c282	build/pkgs/setuptools_scm/SPKG.rst: Reformat title in format 'spkg_name: Short description'
​6f3b864	build/pkgs/[s-t]*/SPKG.rst: Reformat title in format 'spkg_name: Short description'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​23558f8

build/pkgs/[t-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all standard packages

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​17789df

build/pkgs/[z-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all optional packages

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​944408d

build/pkgs/[a-z]*/SPKG.rst: Reformat title in format 'spkg_name: Short description' -- completes coverage of all experimental packages

[Matthias Köppe]

Completed. Ready for review. ./bootstrap && make doc-html will build a much improved local/share/doc/sage/html/en/reference/spkg/index.html

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​feafa60	src/doc/en/installation/standard_packages.rst: Remove
​58c63e1	src/doc/bootstrap: Add brief explanation of the standard/optional/experimental package types

[Dima Pasechnik]

[dochtml] OSError: /mnt/opt/Sage/sage-dev/src/doc/en/reference/spkg/benzene.rst:47: WARNING: Literal block ends without a blank line; unexpected unindent.

indeed, in this generated file one sees

...
Version Information
-------------------

package-version.txt::

    20130630
Equivalent System Packages
--------------------------

...

[Dima Pasechnik]

New commits:

​54d4fb1

extra line after version

[Dima Pasechnik]

lgtm

[Matthias Köppe]

Thanks!

[Matthias Köppe]

New commits:

​2eac7a1	src/doc/bootstrap: Use sage-spkg-info directly instead of 'sage --info', much faster now
​3fe8d4c	src/sage/tests/cmdline.py: Adjust doctests

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​49ad8ae

src/sage/tests/cmdline.py: Adjust doctest properly

[Dima Pasechnik]

OK, good

[Matthias Köppe]

Thanks!