Ticket #5318 (closed enhancement: fixed)
[with patch, positive review] ideas for improving random testers (like #4779)
|Reported by:||cwitty||Owned by:||mabshoff|
Random testers (like the ones in #4779) should have a structure something like this (untested):
def test_foo(n_trials, seed=None, verbose=False): with random_seed(seed): used_seed = initial_seed() try: for i in range(n_trials): # or whatever ... do test ... if verbose, then print out the details of the test you're running except: print "We've detected a failure in random testing." print "Please report this bug; you may be the only person" print "in the world to see this particular problem!" print "initial seed: " + used_seed print "trial: " + i raise
Then the doctests should start with:
sage: test_foo(2, seed=0, verbose=True) ... verbose output from two tests; should always be the same across machines, etc.
to verify that the test is correctly using the randstate framework, so that failures can be reproduced.
Then you can continue to:
sage: test_foo(10) sage: test_foo(100) # long time
which will use truly random seeds (with /dev/urandom).
The above should be adjusted if you want to run the testing function for a very long time; you would want to re-initialize the random seed at least every few seconds, so that if you detect a problem after running for several hours, you don't have to run for the same several hours to reproduce it.
The simplest way is not to loop for long inside the function; instead, do:
while True: test_foo(100)
(That's a lot of boilerplate; maybe this whole setup can be encapsulated in a decorator?)
- Summary changed from ideas for improving random testers (like #4779) to [with patch, needs review] ideas for improving random testers (like #4779)
- Milestone changed from sage-3.4.1 to sage-3.3
- Summary changed from [with patch, needs review] ideas for improving random testers (like #4779) to [with patch, positive review] ideas for improving random testers (like #4779)