Features and optional tags for sage modules provided by separate distributions

[mkoeppe]

... so that we can start writing # optional - sage.symbolic and similar.

We use it in #32432 (sagemath-polyhedra) to skip doctests that depend on sage.graphs, sage.combinat, sage.rings.number_field etc.

[mkoeppe]

Currently optional tags can contain neither . nor -:

sage:         sage: from sage.doctest.parsing import parse_optional_tags
....: 
sage: parse_optional_tags("sage: #optional -- my.p.kg")
{'my'}
sage: parse_optional_tags("sage: #optional -- sagemath-symbolics")
{'sagemath'}

---

New commits:

​9096edb

sage.features.sagemath: New

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2938e13	src/sage/doctest/parsing.py: Allow . in optional tags
​747b458	sage.doctest.control, sage.features.sagemath: Provide/use optional tags

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e3a541e	Fixup
​5e04a85	src/sage/features/sagemath.py: Fixup

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​66cf3e1

src/sage/doctest/control.py: Fixup handling of sage_optional_tags

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1ec0c48

src/sage/features/sagemath.py: Add sage.rings.number_field

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0063749

src/sage/features/sagemath.py: Add features for modules that were optional extensions

[tscrim]

Next round of stupid questions: How is this going to work? Are we doing to have to add many such markers to a lot of doctests? How will we as developers test which ones we need to add?

[tscrim]

Replying to tscrim: Are we doing to have to add many such markers to a lot of doctests?

I guess this part will be alleviated by #30778.

[mkoeppe]

Replying to tscrim:

How will we as developers test which ones we need to add?

By running the test infrastructure of the subset distributions, such as the one added in #32432 (sagemath-polyhedra). This can be done locally, but we will also run this automatically on GH Actions

[jhpalmieri]

Still no hyphens allowed in # optional - keyword? In any case, please also change the documentation in developer/coding_basics.rst to match the behavior (currently says "Any punctuation (periods, commas, hyphens, semicolons, ...) after the first word ends the list of packages."). I think we should have a clearly defined (and well-thought out, or is that too much to ask?) syntax for the keywords. This section of the documentation should ideally also provide examples in which tags are combined, as in # optional - abc, long time. And I guess that is why we chose to parse # optional - abc, xyz as abc rather than abc and xyz. It is a little odd that with the proposed changes, # optional - abc. long time will be treated very differently than # optional - abc, long time.

[mkoeppe]

Replying to jhpalmieri:

Still no hyphens allowed in # optional - keyword?

That's right -- this reflects the fact that our tags for optional packages use the spkg names, which use underscore, not dash. Dashes appear in Python distribution package names (tools like setuptools normalize underscores to dashes).

[mkoeppe]

Replying to jhpalmieri:

It is a little odd that with the proposed changes, # optional - abc. long time will be treated very differently than # optional - abc, long time.

Using . like this has not been observed in the wild, as can be checked with git grep '#.*optional.*[.]'

[mkoeppe]

I'll update the documentation

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​14fd1e5

src/doc/en/developer/coding_basics.rst: Update discussion of feature tags

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​27c53ac

src/sage/features/sagemath.py: Add 'sage.plot'

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​ea548d7	sage.features.four_ti_2: New, use it in sage.interfaces.four_ti_2, sage.sandpiles
​f826ded	build/pkgs/4ti2/spkg-configure.m4: Check for executable's with prefix 4ti2_ too
​56016ce	use AC_LINK_IFELSE instead of obsolete AC_TRY_LINK
​2b45b77	sage.feature.join_feature: New, factored out from LatteFeature; use it to implement FourTi2Feature
​5c23cc9	DocTestReporter: Fix 'sage -t --optional=all'
​1b8634d	sage.doctest.external: Add 4ti2
​d9d4f99	Merge tag '9.4.beta6' into t/30887/public/30887
​646e182	src/sage/features/four_ti_2.py: Move import of SAGE_ENV inside the __init__ method, to remove confusion of sage.misc.dev_tools
​180e31d	Merge #30887
​10e8d63	sage.features.sagemath: Use JoinFeature when tag is different from the actually tested module

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​654d09c

sage.features.sagemath: Change sage_optional_tags to sage_features

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f63a7d0

src/sage/features/: Move features depending on optional packages to separate files

[tscrim]

John, any other comments? I think I am ready to set a positive review.

[jhpalmieri]

I am happy with it. Please go ahead with a positive review when you are ready.

[mkoeppe]

Thanks for the review!

[git]

Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:

​4558791

Merge tag '9.5.beta3' into t/32614/features_and_optional_tags_for_sage_subset_distributions