#29962 closed enhancement (fixed)
Introduce randomseed option to allow fuzzing of doctests
Reported by:  ghkliem  Owned by:  

Priority:  major  Milestone:  sage9.2 
Component:  doctest framework  Keywords:  doctests, random 
Cc:  slelievre  Merged in:  
Authors:  Jonathan Kliem  Reviewers:  Markus Wageringel, Matthias Koeppe 
Report Upstream:  N/A  Work issues:  
Branch:  1d99129 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
This is the first step towards #29935.
We introduce an option for doctests: randomseed
.
This allows specifying which seed to use for tests involving randomness.
The seed is displayed in the test log:
sage t long randomseed=9876543210 src/sage/all.py ... Doctesting 1 file. sage t long randomseed=9876543210 src/sage/all.py [16 tests, 0.73 s]  All tests passed!  Total time for all tests: 0.8 seconds cpu time: 0.7 seconds cumulative wall time: 0.7 seconds
which makes it easy to rerun tests with the same seed.
The seed defaults to 0
for now:
sage t long src/sage/all.py ... Doctesting 1 file. sage t long randomseed=0 src/sage/all.py [16 tests, 0.73 s]  All tests passed!  Total time for all tests: 0.8 seconds cpu time: 0.7 seconds cumulative wall time: 0.7 seconds
but the plan in #29935 is to eventually have the random seed itself picked at random by default.
Change History (23)
comment:1 Changed 19 months ago by
 Branch set to public/29962
 Commit set to 998b1b94ce1289ea92451a86e5f6191c37eaeb5a
 Status changed from new to needs_review
comment:2 Changed 19 months ago by
 Commit changed from 998b1b94ce1289ea92451a86e5f6191c37eaeb5a to b6a5dc7c96125b6140e56feafb82dd13599998e7
Branch pushed to git repo; I updated commit sha1. New commits:
23ed583  fix doctest in hyperbolic_space/hyperbolic_point

5283dc4  use abs tol flag

7b244c0  modify doctests to the extend that they hold with fuzz

228f379  Merge branch 'public/29936' of git://trac.sagemath.org/sage into public/29962

5c7e562  fix double description of hypercube

e1bf211  remove set_random_seed

0e7a998  Merge branch 'public/29904' of git://trac.sagemath.org/sage into public/29962

b6a5dc7  fix random test in geometry/linear_expression

comment:3 Changed 19 months ago by
 Commit changed from b6a5dc7c96125b6140e56feafb82dd13599998e7 to 998b1b94ce1289ea92451a86e5f6191c37eaeb5a
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
comment:4 Changed 19 months ago by
 Commit changed from 998b1b94ce1289ea92451a86e5f6191c37eaeb5a to 1d7b00e3fc2ebc1dc9982a2df91d15e3f12e9432
Branch pushed to git repo; I updated commit sha1. New commits:
1d7b00e  dash instead of underscore for command line options

comment:5 Changed 19 months ago by
 Description modified (diff)
comment:6 Changed 19 months ago by
 Reviewers set to Markus Wageringel
Overall, this looks good to me. The following changes seem to be needed:
 sage: subprocess.call(["sage", "t", "warnlong", "0", "randomseed=0", random_seed.rst"], **kwds) # long time + sage: subprocess.call(["sage", "t", "warnlong", "0", "randomseed=0", "random_seed.rst"], **kwds) # long time
  ``random_seed[=seed]``  random seed for fuzzing doctests +  ``randomseed[=seed]``  random seed for fuzzing doctests
comment:7 Changed 19 months ago by
The branch adds this to the documentation, which is not true for this ticket:
 a/src/doc/en/developer/doctesting.rst +++ b/src/doc/en/developer/doctesting.rst ... +Doctests start from a random seed:: + + [kliem@sage sage9.2]$ sage t src/sage/doctest/tests/random_seed.rst ... + sage t warnlong 89.5 randomseed=112986622569797306072457879734474628454 src/sage/doctest/tests/random_seed.rst + **********************************************************************
comment:8 Changed 19 months ago by
 Branch changed from public/29962 to public/29962reb
 Commit changed from 1d7b00e3fc2ebc1dc9982a2df91d15e3f12e9432 to b62f781647851a3f27ecc95ea4b98b53838da112
comment:9 Changed 19 months ago by
 Reviewers changed from Markus Wageringel to Markus Wageringel, Matthias Koeppe
From my side this is a positive review  if patchbot is green
comment:10 Changed 19 months ago by
 Status changed from needs_review to positive_review
Thanks. This successfully passes ptestlong.
comment:11 Changed 19 months ago by
Thank you.
comment:12 Changed 19 months ago by
 Status changed from positive_review to needs_work
Merge conflict
comment:13 Changed 19 months ago by
 Branch changed from public/29962reb to public/29962reb2
 Commit changed from b62f781647851a3f27ecc95ea4b98b53838da112 to 1d99129f26f4a065f9f9e5e13c3d5120a029e89f
 Status changed from needs_work to needs_review
comment:14 Changed 19 months ago by
 Status changed from needs_review to positive_review
comment:15 Changed 19 months ago by
Thank you.
comment:16 Changed 19 months ago by
 Branch changed from public/29962reb2 to 1d99129f26f4a065f9f9e5e13c3d5120a029e89f
 Resolution set to fixed
 Status changed from positive_review to closed
comment:17 Changed 17 months ago by
 Commit 1d99129f26f4a065f9f9e5e13c3d5120a029e89f deleted
What random seeds are allowed? Any integer? Of any sign? Arbitrarily large?
comment:18 Changed 17 months ago by
This is how we will eventually get the random seeds.
sage: import sage.misc.randstate as randstate sage: randstate.set_random_seed() sage: randstate.initial_seed() 11032378495085541661748859066830408537
At least that's the plan for now. Any random seed that you can feed into set_random_seed
should work.
comment:19 Changed 17 months ago by
The random_seed
docstring says seed "must be coercible to a Python long".
Is that from 2**127
to 2**127  1
?
Or are 32bit and 64bit systems different there?
Also isn't "long" a Python 2 concept, with only "int" existing in Python 3?
Could we document the admissible range in a followup ticket, for nonexperts like me?
comment:20 Changed 17 months ago by
And thanks for the work on varying random seeds when testing!
comment:21 Changed 17 months ago by
I added this issue to #29935.
The idea was that once all parts of Sage are ready for it, the default would be to select a "random" random seed.
comment:22 Changed 9 months ago by
 Description modified (diff)
comment:23 Changed 8 months ago by
 Cc slelievre added
 Description modified (diff)
 Summary changed from Allow fuzzing of doctests to Introduce randomseed option to allow fuzzing of doctests
New commits:
start from a "random" random seed for doctesting
make random seed reproducible
document random_seed
default random seed 0 for now