#29962 closed enhancement (fixed)
Introduce random-seed option to allow fuzzing of doctests
Reported by: | gh-kliem | Owned by: | |
---|---|---|---|
Priority: | major | Milestone: | sage-9.2 |
Component: | doctest framework | Keywords: | doctests, random |
Cc: | slelievre | Merged in: | |
Authors: | Jonathan Kliem | Reviewers: | Markus Wageringel, Matthias Koeppe |
Report Upstream: | N/A | Work issues: | |
Branch: | 1d99129 (Commits, GitHub, GitLab) | Commit: | |
Dependencies: | Stopgaps: |
Description (last modified by )
This is the first step towards #29935.
We introduce an option for doctests: --random-seed
.
This allows specifying which seed to use for tests involving randomness.
The seed is displayed in the test log:
sage -t --long --random-seed=9876543210 src/sage/all.py ... Doctesting 1 file. sage -t --long --random-seed=9876543210 src/sage/all.py [16 tests, 0.73 s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: 0.8 seconds cpu time: 0.7 seconds cumulative wall time: 0.7 seconds
which makes it easy to re-run tests with the same seed.
The seed defaults to 0
for now:
sage -t --long src/sage/all.py ... Doctesting 1 file. sage -t --long --random-seed=0 src/sage/all.py [16 tests, 0.73 s] ---------------------------------------------------------------------- All tests passed! ---------------------------------------------------------------------- Total time for all tests: 0.8 seconds cpu time: 0.7 seconds cumulative wall time: 0.7 seconds
but the plan in #29935 is to eventually have the random seed itself picked at random by default.
Change History (23)
comment:1 Changed 2 years ago by
- Branch set to public/29962
- Commit set to 998b1b94ce1289ea92451a86e5f6191c37eaeb5a
- Status changed from new to needs_review
comment:2 Changed 2 years ago by
- Commit changed from 998b1b94ce1289ea92451a86e5f6191c37eaeb5a to b6a5dc7c96125b6140e56feafb82dd13599998e7
Branch pushed to git repo; I updated commit sha1. New commits:
23ed583 | fix doctest in hyperbolic_space/hyperbolic_point
|
5283dc4 | use abs tol flag
|
7b244c0 | modify doctests to the extend that they hold with fuzz
|
228f379 | Merge branch 'public/29936' of git://trac.sagemath.org/sage into public/29962
|
5c7e562 | fix double description of hypercube
|
e1bf211 | remove set_random_seed
|
0e7a998 | Merge branch 'public/29904' of git://trac.sagemath.org/sage into public/29962
|
b6a5dc7 | fix random test in geometry/linear_expression
|
comment:3 Changed 2 years ago by
- Commit changed from b6a5dc7c96125b6140e56feafb82dd13599998e7 to 998b1b94ce1289ea92451a86e5f6191c37eaeb5a
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
comment:4 Changed 2 years ago by
- Commit changed from 998b1b94ce1289ea92451a86e5f6191c37eaeb5a to 1d7b00e3fc2ebc1dc9982a2df91d15e3f12e9432
Branch pushed to git repo; I updated commit sha1. New commits:
1d7b00e | dash instead of underscore for command line options
|
comment:5 Changed 2 years ago by
- Description modified (diff)
comment:6 Changed 23 months ago by
- Reviewers set to Markus Wageringel
Overall, this looks good to me. The following changes seem to be needed:
- sage: subprocess.call(["sage", "-t", "--warn-long", "0", "--random-seed=0", -random_seed.rst"], **kwds) # long time + sage: subprocess.call(["sage", "-t", "--warn-long", "0", "--random-seed=0", "random_seed.rst"], **kwds) # long time
- - ``--random_seed[=seed]`` -- random seed for fuzzing doctests + - ``--random-seed[=seed]`` -- random seed for fuzzing doctests
comment:7 Changed 23 months ago by
The branch adds this to the documentation, which is not true for this ticket:
--- a/src/doc/en/developer/doctesting.rst +++ b/src/doc/en/developer/doctesting.rst ... +Doctests start from a random seed:: + + [kliem@sage sage-9.2]$ sage -t src/sage/doctest/tests/random_seed.rst ... + sage -t --warn-long 89.5 --random-seed=112986622569797306072457879734474628454 src/sage/doctest/tests/random_seed.rst + **********************************************************************
comment:8 Changed 23 months ago by
- Branch changed from public/29962 to public/29962-reb
- Commit changed from 1d7b00e3fc2ebc1dc9982a2df91d15e3f12e9432 to b62f781647851a3f27ecc95ea4b98b53838da112
comment:9 Changed 23 months ago by
- Reviewers changed from Markus Wageringel to Markus Wageringel, Matthias Koeppe
From my side this is a positive review - if patchbot is green
comment:10 Changed 23 months ago by
- Status changed from needs_review to positive_review
Thanks. This successfully passes ptestlong.
comment:11 Changed 23 months ago by
Thank you.
comment:12 Changed 23 months ago by
- Status changed from positive_review to needs_work
Merge conflict
comment:13 Changed 23 months ago by
- Branch changed from public/29962-reb to public/29962-reb2
- Commit changed from b62f781647851a3f27ecc95ea4b98b53838da112 to 1d99129f26f4a065f9f9e5e13c3d5120a029e89f
- Status changed from needs_work to needs_review
comment:14 Changed 23 months ago by
- Status changed from needs_review to positive_review
comment:15 Changed 23 months ago by
Thank you.
comment:16 Changed 22 months ago by
- Branch changed from public/29962-reb2 to 1d99129f26f4a065f9f9e5e13c3d5120a029e89f
- Resolution set to fixed
- Status changed from positive_review to closed
comment:17 Changed 21 months ago by
- Commit 1d99129f26f4a065f9f9e5e13c3d5120a029e89f deleted
What random seeds are allowed? Any integer? Of any sign? Arbitrarily large?
comment:18 Changed 21 months ago by
This is how we will eventually get the random seeds.
sage: import sage.misc.randstate as randstate sage: randstate.set_random_seed() sage: randstate.initial_seed() 11032378495085541661748859066830408537
At least that's the plan for now. Any random seed that you can feed into set_random_seed
should work.
(Edit: But I wouldn't be surprised if there are bugs somewhere with some strange random seed. There is no way, you can avoid all of them. But at least you can prepare for rather probable cases.)
comment:19 Changed 21 months ago by
The random_seed
docstring says seed "must be coercible to a Python long".
Is that from -2**127
to 2**127 - 1
?
Or are 32-bit and 64-bit systems different there?
Also isn't "long" a Python 2 concept, with only "int" existing in Python 3?
Could we document the admissible range in a follow-up ticket, for non-experts like me?
comment:20 Changed 21 months ago by
And thanks for the work on varying random seeds when testing!
comment:21 Changed 21 months ago by
I added this issue to #29935.
The idea was that once all parts of Sage are ready for it, the default would be to select a "random" random seed.
comment:22 Changed 13 months ago by
- Description modified (diff)
comment:23 Changed 12 months ago by
- Cc slelievre added
- Description modified (diff)
- Summary changed from Allow fuzzing of doctests to Introduce random-seed option to allow fuzzing of doctests
New commits:
start from a "random" random seed for doctesting
make random seed reproducible
document random_seed
default random seed 0 for now