Allow reporting of skipped optional tests

[roed]

From a discussion on ​sage-combinat-devel:

Optionally print extra information after doctesting: how many optional doctests were skipped, and how they were labeled ("skipped 5 doctests marked "optional -- bug").

See also #14276.

Apply 13278.2.patch​

Apply 13278_script.patch​ to the script repo

[roed]

It was necessary to do some cleanup in sage.doctest.sources.DocTestSource: now it just stores an options object rather than separate options that it cares about.

There will be some double counting if optional tests are marked with multiple tags.

For ReST files with inner sources we'll also miss the optional tests in the inner sources.

Both issues seem minor enough that they're not worth the extra code complexity.

[jdemeyer]

Could you add a test to sage/doctest/test.py to test a complete sage -t --show-skipped run (also checking that optional tests which are run are not reported).

And I would enable --show-skipped automatically when --verbose is given.

[roed]

Replying to jdemeyer:

Could you add a test to sage/doctest/test.py to test a complete sage -t --show-skipped run (also checking that optional tests which are run are not reported).

And I would enable --show-skipped automatically when --verbose is given.

Done.

[jdemeyer]

Why still use useabspath here (but remove it from the documentation)?

def __init__(self, path, options, useabspath=False)

especially since you replace

if self.useabspath: 

by

if self.options.abspath: 

[jdemeyer]

And why isn't --long displayed in the output here:

sage: subprocess.call(["sage", "-t", "--long", "--show-skipped", "--optional=sage,gap", "show_skipped.rst"], cwd=tests_dir) 
Running doctests ... 
Doctesting 1 file. 
sage -t show_skipped.rst 

[jdemeyer]

Line 405, you could replace

untested = optionals["not tested"] + optionals["not implemented"]

by

untested = optionals.pop("not tested") + optionals.pop("not implemented")

Then you don't need to check for "not tested" or "not implemented" further.

And I think the translation "known bug" -> "bug" should better be moved inside the parse_optional_tags() function itself.

[roed]

Fixed.

[jdemeyer]

**********************************************************************
File "devel/sage/sage/doctest/test.py", line 240, in sage.doctest.test
Failed example:
    subprocess.call(["sage", "-t", "--long", "--show-skipped", "--optional=sage,gap", "show_skipped.rst"], cwd=tests_dir)
Expected:
    Running doctests ...
    Doctesting 1 file.
    sage -t --long show_skipped.rst
        1 unlabeled test not run
        2 tests not run due to known bugs
        1 other test skipped
        [3 tests, ... s]
    ----------------------------------------------------------------------
    All tests passed!
    ----------------------------------------------------------------------
    ...
    0
Got:
    Running doctests with ID 2013-03-27-00-20-38-1d0ae1fc.
    Doctesting 1 file.
    sage -t --long show_skipped.rst
        1 unlabeled test not run
        2 tests not run due to known bugs
        0 not tested tests not run
        1 other test skipped
        [3 tests, 0.0 s]
    ----------------------------------------------------------------------
    All tests passed!
    ----------------------------------------------------------------------
    Total time for all tests: 0.0 seconds
        cpu time: 0.0 seconds
        cumulative wall time: 0.0 seconds
    0
**********************************************************************

I'll have a look.

[jdemeyer]

13278_review2.patch​ (which replaces 13278_review.patch​) needs review.

[roed]

Works for me.

[jdemeyer]

This conflicts massively with #14331, rebasing... :-(

[jdemeyer]

Rebased and squashed the two patches.