#13278 closed enhancement (fixed)
Allow reporting of skipped optional tests
Reported by: | roed | Owned by: | mvngu |
---|---|---|---|
Priority: | major | Milestone: | sage-5.9 |
Component: | doctest framework | Keywords: | |
Cc: | Merged in: | sage-5.9.beta2 | |
Authors: | David Roe | Reviewers: | Jeroen Demeyer |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | #12415, #14331 | Stopgaps: |
Description (last modified by )
From a discussion on sage-combinat-devel:
Optionally print extra information after doctesting: how many optional doctests were skipped, and how they were labeled ("skipped 5 doctests marked "optional -- bug").
See also #14276.
Apply 13278.2.patch
Apply 13278_script.patch to the script repo
Attachments (5)
Change History (23)
comment:1 Changed 9 years ago by
- Status changed from new to needs_review
Changed 9 years ago by
comment:2 Changed 9 years ago by
- Description modified (diff)
comment:3 Changed 9 years ago by
- Description modified (diff)
comment:4 follow-up: ↓ 5 Changed 9 years ago by
Could you add a test to sage/doctest/test.py
to test a complete sage -t --show-skipped
run (also checking that optional tests which are run are not reported).
And I would enable --show-skipped
automatically when --verbose
is given.
Changed 9 years ago by
comment:5 in reply to: ↑ 4 Changed 9 years ago by
Replying to jdemeyer:
Could you add a test to
sage/doctest/test.py
to test a completesage -t --show-skipped
run (also checking that optional tests which are run are not reported).And I would enable
--show-skipped
automatically when--verbose
is given.
Done.
comment:6 Changed 9 years ago by
Why still use useabspath
here (but remove it from the documentation)?
def __init__(self, path, options, useabspath=False)
especially since you replace
if self.useabspath:
by
if self.options.abspath:
comment:7 Changed 9 years ago by
And why isn't --long
displayed in the output here:
sage: subprocess.call(["sage", "-t", "--long", "--show-skipped", "--optional=sage,gap", "show_skipped.rst"], cwd=tests_dir) Running doctests ... Doctesting 1 file. sage -t show_skipped.rst
comment:8 Changed 9 years ago by
Line 405, you could replace
untested = optionals["not tested"] + optionals["not implemented"]
by
untested = optionals.pop("not tested") + optionals.pop("not implemented")
Then you don't need to check for "not tested" or "not implemented" further.
And I think the translation "known bug" -> "bug" should better be moved inside the parse_optional_tags()
function itself.
comment:9 Changed 9 years ago by
- Reviewers set to Jeroen Demeyer
- Status changed from needs_review to needs_work
comment:10 Changed 9 years ago by
- Description modified (diff)
- Status changed from needs_work to needs_review
Fixed.
Changed 9 years ago by
comment:11 Changed 9 years ago by
- Status changed from needs_review to needs_work
********************************************************************** File "devel/sage/sage/doctest/test.py", line 240, in sage.doctest.test Failed example: subprocess.call(["sage", "-t", "--long", "--show-skipped", "--optional=sage,gap", "show_skipped.rst"], cwd=tests_dir) Expected: Running doctests ... Doctesting 1 file. sage -t --long show_skipped.rst 1 unlabeled test not run 2 tests not run due to known bugs 1 other test skipped [3 tests, ... s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- ... 0 Got: Running doctests with ID 2013-03-27-00-20-38-1d0ae1fc. Doctesting 1 file. sage -t --long show_skipped.rst 1 unlabeled test not run 2 tests not run due to known bugs 0 not tested tests not run 1 other test skipped [3 tests, 0.0 s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: 0.0 seconds cpu time: 0.0 seconds cumulative wall time: 0.0 seconds 0 **********************************************************************
I'll have a look.
comment:12 Changed 9 years ago by
- Dependencies changed from #12415 to #12415, #14331
Changed 9 years ago by
comment:13 Changed 9 years ago by
- Description modified (diff)
- Status changed from needs_work to needs_review
13278_review2.patch (which replaces 13278_review.patch) needs review.
comment:15 Changed 9 years ago by
This conflicts massively with #14331, rebasing... :-(
Changed 9 years ago by
comment:17 Changed 9 years ago by
- Merged in set to sage-5.9.beta2
- Resolution set to fixed
- Status changed from positive_review to closed
comment:18 Changed 9 years ago by
- Component changed from doctest to doctest framework
It was necessary to do some cleanup in
sage.doctest.sources.DocTestSource
: now it just stores an options object rather than separate options that it cares about.There will be some double counting if optional tests are marked with multiple tags.
For ReST files with inner sources we'll also miss the optional tests in the inner sources.
Both issues seem minor enough that they're not worth the extra code complexity.