Opened 11 years ago
Closed 8 years ago
#9225 closed enhancement (invalid)
New doctesting features
Reported by: | mpatel | Owned by: | mvngu |
---|---|---|---|
Priority: | major | Milestone: | sage-duplicate/invalid/wontfix |
Component: | doctest coverage | Keywords: | |
Cc: | wjp, leif, rbeezer | Merged in: | |
Authors: | Reviewers: | David Roe, Jeroen Demeyer | |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | Stopgaps: |
Description (last modified by )
This is an umbrella ticket for various improvements to the Sage doctesting framework. For example:
When we doctest several files in parallel or in sequence, it might be helpful to print stats relating to the test suite's progress, e.g.,
$ sage -t monoids/ sage -t "monoids/monoid.py" 1/10 1.9 s / 1.9 s sage -t "monoids/free_monoid.py" 2/10 2.0 s / 3.9 s [...]What other columns would be useful? Coverage?
Please see the comments for more ideas. Suggestions are welcome!
Change History (19)
comment:1 Changed 11 years ago by
- Cc wjp added
comment:2 follow-up: ↓ 11 Changed 11 years ago by
comment:3 Changed 11 years ago by
- Cc leif added
comment:4 Changed 11 years ago by
- Priority changed from minor to major
Other possibilities, probably mentioned before and probably for other tickets:
- "Librarify:" Make it possible to doctest files and objects from the Sage command-line or notebook, e.g.,
sage: doctest('file.sage', long=True, optional=['axiom', 'magma']) sage: doctest(os.path.join(SAGE_ROOT, 'devel', 'sage', 'sage', 'monoids'), processes=4) sage: stats = {} # Collect errors, counts, timings, etc. sage: doctest(['foo.py', 'bar.pyx'], stats=stats) sage: def f(): ....: """ ....: sage: f() ....: 1 ....: """ ....: return 1 ....: sage: doctest(f)
- Doctest the doctesting framework! I think we could include this example, at least.
- An option to run an individual
file.py
's "examples" (i.e., the filesexample_*
functions) in parallel.
comment:5 follow-up: ↓ 8 Changed 11 years ago by
A way to upload anonymized test stats and system information to a remote server?
comment:6 follow-up: ↓ 9 Changed 11 years ago by
Also potentially useful: doctesting the same file multiple times in parallel. I think this is just a matter of generating in sage-doctest
a unique name, e.g., .doctest_foo-R3X87S.py
for each run on foo.py
.
comment:7 Changed 11 years ago by
I have a suspicion that some doctests fail when the system is under heavy load. It would be good if the one-minute and perhaps 5 minute load average could be recorded with any failures, along with the dates and time as I mentioned before.
comment:8 in reply to: ↑ 5 Changed 11 years ago by
Replying to mpatel:
A way to upload anonymized test stats and system information to a remote server?
Yes, but with the option of giving a name if one wishes to. It will make problems easier to manage if people did give a name + email, but certainly make it optional.
comment:9 in reply to: ↑ 6 Changed 11 years ago by
comment:10 Changed 11 years ago by
- Description modified (diff)
- Summary changed from Indicate progress and elapsed time when running multiple doctests to New doctesting features
comment:11 in reply to: ↑ 2 ; follow-up: ↓ 12 Changed 11 years ago by
comment:12 in reply to: ↑ 11 Changed 11 years ago by
Replying to drkirkby:
Replying to mpatel:
If we widen the scope here a bit: We should use optparse, at least, to handle command-line arguments.
Though if you look at that link, it says its been depreciated in favor of
argparse
.
That's why I inserted "at least," but I should have more specific. We'll need to upgrade to Python 2.7 to use argparse readily. argparse used to be an independent project.
comment:13 Changed 10 years ago by
- Cc rbeezer added
Rob, perhaps you could state your personal wishes here, too.
(E.g. ./sage -testall ...
with an option to specify the number of threads IIRC rather than always doing that serially.)
comment:14 Changed 10 years ago by
- Description modified (diff)
#10458 improves cut & paste of doctest code & output from an actual Sage session, in that it allows also "....:
" for line continuations. (Previously, for some reason only "...
" was supported in docstrings; also the confusion with an ellipsis intended to mean arbitrary output should be reduced by this ticket.)
Another problem still has to be addressed: The original (non-preparsed) source code of continuation lines gets "lost", i.e. doesn't end up in the end-of-line comments of the generated test files (.doctest_*
) as it does for ordinary lines, hence also won't be shown on doctest errors or when one doctests in verbose mode.
comment:15 Changed 9 years ago by
- Description modified (diff)
comment:16 Changed 9 years ago by
#12415 will add many of these features.
comment:17 Changed 8 years ago by
- Milestone set to sage-duplicate/invalid/wontfix
- Reviewers set to David Roe, Jeroen Demeyer
- Status changed from new to needs_review
As David mentioned, there is a lot of work at #12415. And apart from that, this ticket is extremely vague.
comment:18 Changed 8 years ago by
- Status changed from needs_review to positive_review
comment:19 Changed 8 years ago by
- Resolution set to invalid
- Status changed from positive_review to closed
If we widen the scope here a bit: We should use optparse, at least, to handle command-line arguments.