Opened 6 years ago
Closed 6 years ago
#22832 closed enhancement (fixed)
Add an option to the test runner to exit immediately upon the first failure
Reported by: | Erik Bray | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-8.0 |
Component: | doctest framework | Keywords: | |
Cc: | Merged in: | ||
Authors: | Erik Bray | Reviewers: | Jeroen Demeyer |
Report Upstream: | N/A | Work issues: | |
Branch: | c6a5b40 (Commits, GitHub, GitLab) | Commit: | c6a5b40146506795873a1cb7b64e645258251a90 |
Dependencies: | Stopgaps: |
Description (last modified by )
Inspired by the --exitfirst
option to py.test
, this adds a --exitfirst
option to Sage's test runner to exit immediately upon the first test failure.
I've wanted something like this at several points, but one reason in particular is that part of the build of the patchbot Docker image is to perform the baseline test run, so that the image itself can be reused without having to rebuild the baseline each time. Obviously, if a test fails during the baseline test run then I want the Docker image build to fail immediately, and not wait until the rest of the test suite has run.
Change History (11)
comment:1 Changed 6 years ago by
Authors: | → Erik Bray |
---|---|
Branch: | → u/embray/tests/fail-once |
Commit: | → 2f359e8a5e5da8b8dba9475e7b168b2cf74c0e65 |
Status: | new → needs_review |
comment:2 follow-up: 7 Changed 6 years ago by
Replying to embray:
Inspired by the
--exitfirst
option topy.test
, this adds a--fail-once
option to Sage's test runner to exit immediately upon the first test failure.
Obvious question: why not call the Sage option exitfirst
? I find it a better name than fail-once
.
comment:3 Changed 6 years ago by
Status: | needs_review → needs_work |
---|
I'm making some changes, hang on...
comment:4 Changed 6 years ago by
Branch: | u/embray/tests/fail-once → u/jdemeyer/tests/fail-once |
---|
comment:5 Changed 6 years ago by
Commit: | 2f359e8a5e5da8b8dba9475e7b168b2cf74c0e65 → c2bff84e640507306d52b108e3ee440eb555ddba |
---|---|
Description: | modified (diff) |
Reviewers: | → Jeroen Demeyer |
Status: | needs_work → needs_review |
comment:7 Changed 6 years ago by
Replying to jdemeyer:
Replying to embray:
Inspired by the
--exitfirst
option topy.test
, this adds a--fail-once
option to Sage's test runner to exit immediately upon the first test failure.Obvious question: why not call the Sage option
exitfirst
? I find it a better name thanfail-once
.
I personally find "fail once" to be clearer than "exit first"--"Exit first what?" is what it makes me ask. That said, the documentation makes it clear enough, and I don't mind the consistency.
comment:8 Changed 6 years ago by
Status: | positive_review → needs_work |
---|
Oops, one little nitpick--you forgot to remove the line fail_immediately=False
after renaming it to abort_now
everywhere else.
comment:9 Changed 6 years ago by
Commit: | c2bff84e640507306d52b108e3ee440eb555ddba → c6a5b40146506795873a1cb7b64e645258251a90 |
---|
Branch pushed to git repo; I updated commit sha1. New commits:
c6a5b40 | Remove forgotten line
|
comment:10 Changed 6 years ago by
Status: | needs_work → positive_review |
---|
comment:11 Changed 6 years ago by
Branch: | u/jdemeyer/tests/fail-once → c6a5b40146506795873a1cb7b64e645258251a90 |
---|---|
Resolution: | → fixed |
Status: | positive_review → closed |
New commits:
Initial attempt to add a test runner option to stop a test run as soon as a single failure is encountered.
Make --fail-once work with parallel doctests.