Opened 6 years ago
Closed 6 years ago
#22832 closed enhancement (fixed)
Add an option to the test runner to exit immediately upon the first failure
| Reported by: | Erik Bray | Owned by: | |
|---|---|---|---|
| Priority: | major | Milestone: | sage-8.0 |
| Component: | doctest framework | Keywords: | |
| Cc: | Merged in: | ||
| Authors: | Erik Bray | Reviewers: | Jeroen Demeyer |
| Report Upstream: | N/A | Work issues: | |
| Branch: | c6a5b40 (Commits, GitHub, GitLab) | Commit: | c6a5b40146506795873a1cb7b64e645258251a90 |
| Dependencies: | Stopgaps: |
Status badges
Description (last modified by )
Inspired by the --exitfirst option to py.test, this adds a --exitfirst option to Sage's test runner to exit immediately upon the first test failure.
I've wanted something like this at several points, but one reason in particular is that part of the build of the patchbot Docker image is to perform the baseline test run, so that the image itself can be reused without having to rebuild the baseline each time. Obviously, if a test fails during the baseline test run then I want the Docker image build to fail immediately, and not wait until the rest of the test suite has run.
Change History (11)
comment:1 Changed 6 years ago by
| Authors: | → Erik Bray |
|---|---|
| Branch: | → u/embray/tests/fail-once |
| Commit: | → 2f359e8a5e5da8b8dba9475e7b168b2cf74c0e65 |
| Status: | new → needs_review |
comment:2 follow-up: 7 Changed 6 years ago by
Replying to embray:
Inspired by the
--exitfirstoption topy.test, this adds a--fail-onceoption to Sage's test runner to exit immediately upon the first test failure.
Obvious question: why not call the Sage option exitfirst? I find it a better name than fail-once.
comment:3 Changed 6 years ago by
| Status: | needs_review → needs_work |
|---|
I'm making some changes, hang on...
comment:4 Changed 6 years ago by
| Branch: | u/embray/tests/fail-once → u/jdemeyer/tests/fail-once |
|---|
comment:5 Changed 6 years ago by
| Commit: | 2f359e8a5e5da8b8dba9475e7b168b2cf74c0e65 → c2bff84e640507306d52b108e3ee440eb555ddba |
|---|---|
| Description: | modified (diff) |
| Reviewers: | → Jeroen Demeyer |
| Status: | needs_work → needs_review |
comment:7 Changed 6 years ago by
Replying to jdemeyer:
Replying to embray:
Inspired by the
--exitfirstoption topy.test, this adds a--fail-onceoption to Sage's test runner to exit immediately upon the first test failure.Obvious question: why not call the Sage option
exitfirst? I find it a better name thanfail-once.
I personally find "fail once" to be clearer than "exit first"--"Exit first what?" is what it makes me ask. That said, the documentation makes it clear enough, and I don't mind the consistency.
comment:8 Changed 6 years ago by
| Status: | positive_review → needs_work |
|---|
Oops, one little nitpick--you forgot to remove the line fail_immediately=False after renaming it to abort_now everywhere else.
comment:9 Changed 6 years ago by
| Commit: | c2bff84e640507306d52b108e3ee440eb555ddba → c6a5b40146506795873a1cb7b64e645258251a90 |
|---|
Branch pushed to git repo; I updated commit sha1. New commits:
| c6a5b40 | Remove forgotten line
|
comment:10 Changed 6 years ago by
| Status: | needs_work → positive_review |
|---|
comment:11 Changed 6 years ago by
| Branch: | u/jdemeyer/tests/fail-once → c6a5b40146506795873a1cb7b64e645258251a90 |
|---|---|
| Resolution: | → fixed |
| Status: | positive_review → closed |
New commits:
Initial attempt to add a test runner option to stop a test run as soon as a single failure is encountered.Make --fail-once work with parallel doctests.