Meta-ticket: Add user packages from https://wiki.sagemath.org/SageMathExternalPackages as optional/experimental packages

[mkoeppe]

At least some of the packages listed at ​https://wiki.sagemath.org/SageMathExternalPackages are probably not in worse shape than many of our current optional/experimental packages.

By including them as optional/experimental packages in build/pkgs:

• they would be automatically included in our reference manual and Sage website (see #29655)
• The GH Actions workflows tox-optional.yml and tox-experimental.yml will run them, providing continuous integration that will allow use to catch unintended breaking changes during the Sage development cycle

In this ticket, only pip-installable packages listed on PyPI will be added.

This adds an incentive to package authors to bring their packages to this form.

The packages will be added as "pip" packages instead of "normal packages", ​https://doc.sagemath.org/html/en/developer/packaging.html#package-source-types These do not come with tarball information and do not have to pin the version, so by default the latest version on PyPI would be installed. Hence there is no additional maintenance burden from updating the packages.

Adding the packages is made easy by the new sage -package commands described in ​https://wiki.sagemath.org/ReleaseTours/sage-9.3#For_developers:_Setting_up_Python_packages_from_PyPI_as_Sage_packages

Tickets:

• #31176 Add packages from 3-manifolds / computop (#31180: snappy)
• #31223 sage-train-track
• #31224 surface-dynamics
• #31235 sage-numerical-interactive-mip, cutgeneratingfunctionology
• #31237 slabbe
• #31238 admcycles
• #31239 ore_algebra
• #31240 sage-flatsurf
• #31416 multipolynomial-bases

Discussions:

• ​https://groups.google.com/g/sage-devel/c/ZNyLBKmc0EE/m/oUll3liBAwAJ - on version metadata

[vdelecroix]

Before proceeding, I would like to understand what package developers and package users would gain by declaring Python packages as sage optional packages. For now, I would only consider doing this if

• package versions were not tight to sage versions. In the curent setup a package upgrade needs a ticket review and, when merged, has to wait for a new sage version.
• there is no serious continuous integration for packages (as they have eg for gap)

[mkoeppe]

Replying to vdelecroix:

Before proceeding, I would like to understand what package developers and package users would gain by declaring Python packages as sage optional packages. For now, I would only consider doing this if

• package versions were not tight to sage versions. In the curent setup a package upgrade needs a ticket review and, when merged, has to wait for a new sage version.

The solution here is to use pip packages instead of normal packages. ​https://doc.sagemath.org/html/en/developer/packaging.html#package-source-types

These do not come with tarball information and do not have to pin the version, so by default the latest version on PyPI would be installed.

• there is no serious continuous integration for packages (as they have eg for gap)

Well, we do have workflows on GH Actions that at least tries to install all optional and experimental packages!

• ​https://github.com/sagemath/sage/actions/runs/447854806
• ​https://github.com/sagemath/sage/actions/runs/447854801

[mkoeppe]

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.