Ticket #12415: 12415_doc.patch

File 12415_doc.patch, 59.2 KB (added by jdemeyer, 7 years ago)

Updates documention

  • doc/en/developer/doctesting.rst

    # HG changeset patch
    # User David Roe <roed.math@gmail.com>
    # Date 1342314181 14400
    # Node ID eee466f22f8ab4a9d43243c91ad22c6abb57e187
    # Parent  90e164fd86b376b3b07efa57af3a0fa4d255a4d9
    #12415: Update the developer guide for the new doctesting code.
    
    diff --git a/doc/en/developer/doctesting.rst b/doc/en/developer/doctesting.rst
    a b  
    1111multiple threads. After compiling a source version of Sage, doctesting
    1212can be run on the whole Sage library, on all modules under a given
    1313directory, or on a specified module only. For the purposes of this
    14 chapter, suppose we have compiled Sage 4.8 from source and the top
     14chapter, suppose we have compiled Sage 5.9 from source and the top
    1515level Sage directory is
    1616
    1717::
    1818
    19     [jdemeyer@sage sage-4.8]$ pwd
    20     /scratch/jdemeyer/build/sage-4.8
     19    [jdemeyer@sage sage-5.9]$ pwd
     20    /scratch/jdemeyer/build/sage-5.9
    2121
    2222See the section :ref:`chapter-testing` for information on Sage's
    2323automated testing process. The general syntax for doctesting is as
     
    2626
    2727    /path/to/sage-x.y.z/sage -t [--long] /path/to/sage-x.y.z/path/to/module.py[x]
    2828
    29 where ``--long`` is an optional argument. The version of ``sage`` used must
    30 match the version of Sage containing the module we want to doctest. A
    31 Sage module can be either a Python script (with the file extension
    32 ".py") or it can be a Cython script, in which case it has the file
    33 extension ".pyx".
     29where ``--long`` is an optional argument (see :ref:`section-options`
     30for more options). The version of ``sage`` used must match the version
     31of Sage containing the module we want to doctest. A Sage module can be
     32either a Python script (with the file extension ".py") or it can be a
     33Cython script, in which case it has the file extension ".pyx".
    3434
    3535
    3636Testing a module
     
    4141top level Sage directory of our local Sage installation. Now  we can
    4242start doctesting as demonstrated in the following terminal session::
    4343
    44     [jdemeyer@sage sage-4.8]$ ./sage -t devel/sage-main/sage/games/sudoku.py
    45     sage -t  "devel/sage-main/sage/games/sudoku.py"
    46              [7.3 s]
    47 
    48     ----------------------------------------------------------------------
     44    [jdemeyer@sage sage-5.9]$ ./sage -t devel/sage-main/sage/games/sudoku.py
     45    Running doctests with ID 2012-07-03-03-36-49-d82849c6.
     46    Doctesting 1 file.
     47    sage -t devel/sage-main/sage/games/sudoku.py
     48        [103 tests, 3.6 s]
     49    ------------------------------------------------------------------------
    4950    All tests passed!
    50     Total time for all tests: 7.3 seconds
     51    ------------------------------------------------------------------------
     52    Total time for all tests: 4.8 seconds
     53        cpu time: 3.6 seconds
     54        cumulative wall time: 3.6 seconds
    5155
    5256The numbers output by the test show that testing the sudoku module
    53 takes about six seconds, while testing all specified modules took the
    54 same amount of time. In this case, we only tested one module so it is
    55 not surprising that the total testing time is approximately the same
    56 as the time required to test only that one module. Notice that the
    57 syntax is ::
     57takes about four seconds, while testing all specified modules took the
     58same amount of time; the total time required includes some startup
     59time for the code that runs the tests. In this case, we only tested
     60one module so it is not surprising that the total testing time is
     61approximately the same as the time required to test only that one
     62module. Notice that the syntax is ::
    5863
    59     [jdemeyer@sage sage-4.8]$ ./sage -t devel/sage-main/sage/games/sudoku.py
    60     sage -t  "devel/sage-main/sage/games/sudoku.py"
    61              [7.3 s]
    62 
    63     ----------------------------------------------------------------------
     64    [jdemeyer@sage sage-5.9]$ ./sage -t devel/sage-main/sage/games/sudoku.py
     65    Running doctests with ID 2012-07-03-03-39-02-da6accbb.
     66    Doctesting 1 file.
     67    sage -t devel/sage-main/sage/games/sudoku.py
     68        [103 tests, 3.6 s]
     69    ------------------------------------------------------------------------
    6470    All tests passed!
    65     Total time for all tests: 7.3 seconds
    66     [jdemeyer@sage sage-4.8]$ ./sage -t "devel/sage-main/sage/games/sudoku.py"
    67     sage -t  "devel/sage-main/sage/games/sudoku.py"
    68              [7.5 s]
    69 
    70     ----------------------------------------------------------------------
     71    ------------------------------------------------------------------------
     72    Total time for all tests: 4.9 seconds
     73    [jdemeyer@sage sage-5.9]$ ./sage -t "devel/sage-main/sage/games/sudoku.py
     74    Running doctests with ID 2012-07-03-03-39-02-da6accbb.
     75    Doctesting 1 file.
     76    sage -t devel/sage-main/sage/games/sudoku.py
     77        [103 tests, 3.6 s]
     78    ------------------------------------------------------------------------
    7179    All tests passed!
    72     Total time for all tests: 7.6 seconds
     80    ------------------------------------------------------------------------
     81    Total time for all tests: 4.9 seconds
     82        cpu time: 3.6 seconds
     83        cumulative wall time: 3.6 seconds
    7384
    7485but not
    7586
    7687::
    7788
    78     [jdemeyer@sage sage-4.8]$ ./sage -t sage/games/sudoku.py
    79     ERROR: File ./sage/games/sudoku.py is missing
    80     exit code: 1
    81 
    82     ----------------------------------------------------------------------
    83     The following tests failed:
    84 
    85     ./sage/games/sudoku.py
    86     Total time for all tests: 0.0 seconds
    87     [jdemeyer@sage sage-4.8]$ ./sage -t "sage/games/sudoku.py"
    88     ERROR: File ./sage/games/sudoku.py is missing
    89 
    90     ----------------------------------------------------------------------
    91     The following tests failed:
    92 
    93 
    94             ./sage/games/sudoku.py # File not found
    95     Total time for all tests: 0.0 seconds
     89    [jdemeyer@sage sage-5.9]$ ./sage -t sage/games/sudoku.py
     90    Running doctests with ID 2012-07-03-03-40-53-6cc4f29f.
     91    No files matching sage/games/sudoku.py
     92    No files to doctest
    9693
    9794We can also first ``cd`` to the directory containing the module
    9895``sudoku.py`` and doctest that module as follows::
    9996
    100     [jdemeyer@sage sage-4.8]$ cd devel/sage-main/sage/games/
     97    [jdemeyer@sage sage-5.9]$ cd devel/sage-main/sage/games/
    10198    [jdemeyer@sage games]$ ls
    10299    __init__.py  hexad.py       sudoku.py           sudoku_backtrack.pyx
    103100    all.py       quantumino.py  sudoku_backtrack.c
    104101    [jdemeyer@sage games]$ ../../../../sage -t sudoku.py
    105     sage -t  "devel/sage-main/sage/games/sudoku.py"
    106              [7.1 s]
     102    Running doctests with ID 2012-07-03-03-41-39-95ebd2ff.
     103    Doctesting 1 file.
     104    sage -t sudoku.py
     105        [103 tests, 3.6 s]
     106    ------------------------------------------------------------------------
     107    All tests passed!
     108    ------------------------------------------------------------------------
     109    Total time for all tests: 5.2 seconds
     110        cpu time: 3.6 seconds
     111        cumulative wall time: 3.6 seconds
    107112
    108     ----------------------------------------------------------------------
    109     All tests passed!
    110     Total time for all tests: 7.1 seconds
    111113
    112114In all of the above terminal sessions, we used a local installation of
    113115Sage to test its own modules. Even if we have a system-wide Sage
     
    128130syntax is acceptable because we explicitly specify the Sage
    129131installation in the current ``SAGE_ROOT``::
    130132
    131     [jdemeyer@sage sage-4.8]$ ./sage -t devel/sage-main/sage/games/sudoku.py
    132     ./sage -t "devel/sage-main/sage/games/sudoku.py"
    133              [6.9 s]
    134 
    135     ----------------------------------------------------------------------
     133    [jdemeyer@sage sage-5.9]$ ./sage -t devel/sage-main/sage/games/sudoku.py
     134    Running doctests with ID 2012-07-03-03-43-24-a3449f54.
     135    Doctesting 1 file.
     136    sage -t devel/sage-main/sage/games/sudoku.py
     137        [103 tests, 3.6 s]
     138    ------------------------------------------------------------------------
    136139    All tests passed!
    137     Total time for all tests: 6.9 seconds
    138     [jdemeyer@sage sage-4.8]$ ./sage -t "devel/sage-main/sage/games/sudoku.py"
    139     sage -t  "devel/sage-main/sage/games/sudoku.py"
    140              [7.7 s]
    141 
    142     ----------------------------------------------------------------------
     140    ------------------------------------------------------------------------
     141    Total time for all tests: 4.9 seconds
     142        cpu time: 3.6 seconds
     143        cumulative wall time: 3.6 seconds
     144    [jdemeyer@sage sage-5.9]$ ./sage -t "devel/sage-main/sage/games/sudoku.py"
     145    Running doctests with ID 2012-07-03-03-43-54-ac8ca007.
     146    Doctesting 1 file.
     147    sage -t devel/sage-main/sage/games/sudoku.py
     148        [103 tests, 3.6 s]
     149    ------------------------------------------------------------------------
    143150    All tests passed!
    144     Total time for all tests: 7.7 seconds
     151    ------------------------------------------------------------------------
     152    Total time for all tests: 4.9 seconds
     153        cpu time: 3.6 seconds
     154        cumulative wall time: 3.6 seconds
    145155
    146156The following syntax is not recommended as we are using a system-wide
    147157Sage installation (if it exists):
     
    150160
    151161::
    152162
    153     [jdemeyer@sage sage-4.8]$ sage -t devel/sage-main/sage/games/sudoku.py
     163    [jdemeyer@sage sage-5.9]$ sage -t devel/sage-main/sage/games/sudoku.py
    154164    sage -t  "devel/sage-main/sage/games/sudoku.py"
    155165    **********************************************************************
    156     File "/home/jdemeyer/sage/sage-4.8/devel/sage-main/sage/games/sudoku.py", line 515:
     166    File "/home/jdemeyer/sage/sage-5.9/devel/sage-main/sage/games/sudoku.py", line 515:
    157167        sage: h.solve(algorithm='backtrack').next()
    158168    Exception raised:
    159169        Traceback (most recent call last):
     
    209219all the modules under ``sage/crypto/``. We can use a syntax similar to
    210220that shown above to achieve this::
    211221
    212     [jdemeyer@sage sage-4.8]$ ./sage -t devel/sage-main/sage/crypto/
    213     sage -t  "devel/sage-main/sage/crypto/block_cipher/__init__.py"
    214              [0.1 s]
    215     sage -t  "devel/sage-main/sage/crypto/block_cipher/miniaes.py"
    216              [5.5 s]
    217     sage -t  "devel/sage-main/sage/crypto/block_cipher/all.py"
    218              [0.1 s]
    219     sage -t  "devel/sage-main/sage/crypto/block_cipher/sdes.py"
    220              [4.2 s]
    221     sage -t  "devel/sage-main/sage/crypto/__init__.py"
    222              [0.1 s]
    223     sage -t  "devel/sage-main/sage/crypto/stream.py"
    224              [3.7 s]
    225     sage -t  "devel/sage-main/sage/crypto/classical_cipher.py"
    226              [5.1 s]
    227     sage -t  "devel/sage-main/sage/crypto/boolean_function.pyx"
    228              [7.3 s]
    229     sage -t  "devel/sage-main/sage/crypto/lattice.py"
    230              [3.7 s]
    231     sage -t  "devel/sage-main/sage/crypto/util.py"
    232              [3.4 s]
    233     sage -t  "devel/sage-main/sage/crypto/cryptosystem.py"
    234              [3.6 s]
    235     sage -t  "devel/sage-main/sage/crypto/all.py"
    236              [0.1 s]
    237     sage -t  "devel/sage-main/sage/crypto/mq/__init__.py"
    238              [0.1 s]
    239     sage -t  "devel/sage-main/sage/crypto/mq/sbox.py"
    240              [4.4 s]
    241     sage -t  "devel/sage-main/sage/crypto/mq/mpolynomialsystem.py"
    242              [12.8 s]
    243     sage -t  "devel/sage-main/sage/crypto/mq/sr.py"
    244              [10.6 s]
    245     sage -t  "devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py"
    246              [3.4 s]
    247     sage -t  "devel/sage-main/sage/crypto/cipher.py"
    248              [3.4 s]
    249     sage -t  "devel/sage-main/sage/crypto/classical.py"
    250              [13.8 s]
    251     sage -t  "devel/sage-main/sage/crypto/public_key/blum_goldwasser.py"
    252              [3.5 s]
    253     sage -t  "devel/sage-main/sage/crypto/public_key/__init__.py"
    254              [0.1 s]
    255     sage -t  "devel/sage-main/sage/crypto/public_key/all.py"
    256              [0.1 s]
    257     sage -t  "devel/sage-main/sage/crypto/stream_cipher.py"
    258              [3.4 s]
    259     sage -t  "devel/sage-main/sage/crypto/lfsr.py"
    260              [3.5 s]
    261 
    262     ----------------------------------------------------------------------
     222    [jdemeyer@sage sage-5.9]$ ./sage -t devel/sage-main/sage/crypto
     223    Running doctests with ID 2012-07-03-03-45-40-7f837dcf.
     224    Doctesting 24 files.
     225    sage -t devel/sage-main/sage/crypto/__init__.py
     226        [0 tests, 0.0 s]
     227    sage -t devel/sage-main/sage/crypto/all.py
     228        [0 tests, 0.0 s]
     229    sage -t devel/sage-main/sage/crypto/boolean_function.pyx
     230        [252 tests, 4.4 s]
     231    sage -t devel/sage-main/sage/crypto/cipher.py
     232        [10 tests, 0.0 s]
     233    sage -t devel/sage-main/sage/crypto/classical.py
     234        [718 tests, 11.3 s]
     235    sage -t devel/sage-main/sage/crypto/classical_cipher.py
     236        [130 tests, 0.5 s]
     237    sage -t devel/sage-main/sage/crypto/cryptosystem.py
     238        [82 tests, 0.1 s]
     239    sage -t devel/sage-main/sage/crypto/lattice.py
     240        [1 tests, 0.0 s]
     241    sage -t devel/sage-main/sage/crypto/lfsr.py
     242        [31 tests, 0.1 s]
     243    sage -t devel/sage-main/sage/crypto/stream.py
     244        [17 tests, 0.1 s]
     245    sage -t devel/sage-main/sage/crypto/stream_cipher.py
     246        [114 tests, 0.2 s]
     247    sage -t devel/sage-main/sage/crypto/util.py
     248        [122 tests, 0.2 s]
     249    sage -t devel/sage-main/sage/crypto/block_cipher/__init__.py
     250        [0 tests, 0.0 s]
     251    sage -t devel/sage-main/sage/crypto/block_cipher/all.py
     252        [0 tests, 0.0 s]
     253    sage -t devel/sage-main/sage/crypto/block_cipher/miniaes.py
     254        [430 tests, 1.3 s]
     255    sage -t devel/sage-main/sage/crypto/block_cipher/sdes.py
     256        [290 tests, 0.9 s]
     257    sage -t devel/sage-main/sage/crypto/mq/__init__.py
     258        [0 tests, 0.0 s]
     259    sage -t devel/sage-main/sage/crypto/mq/mpolynomialsystem.py
     260        [320 tests, 9.1 s]
     261    sage -t devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py
     262        [42 tests, 0.1 s]
     263    sage -t devel/sage-main/sage/crypto/mq/sbox.py
     264        [124 tests, 0.8 s]
     265    sage -t devel/sage-main/sage/crypto/mq/sr.py
     266        [435 tests, 5.5 s]
     267    sage -t devel/sage-main/sage/crypto/public_key/__init__.py
     268        [0 tests, 0.0 s]
     269    sage -t devel/sage-main/sage/crypto/public_key/all.py
     270        [0 tests, 0.0 s]
     271    sage -t devel/sage-main/sage/crypto/public_key/blum_goldwasser.py
     272        [135 tests, 0.2 s]
     273    ------------------------------------------------------------------------
    263274    All tests passed!
    264     Total time for all tests: 96.1 seconds
     275    ------------------------------------------------------------------------
     276    Total time for all tests: 38.1 seconds
     277        cpu time: 29.8 seconds
     278        cumulative wall time: 35.1 seconds
    265279
    266280Now we do the same thing, but this time we also use the optional
    267281argument ``--long``::
    268282
    269     [jdemeyer@sage sage-4.8]$ ./sage -t --long devel/sage-main/sage/crypto/
    270     sage -t --long "devel/sage-main/sage/crypto/block_cipher/__init__.py"
    271              [0.1 s]
    272     sage -t --long "devel/sage-main/sage/crypto/block_cipher/miniaes.py"
    273              [4.1 s]
    274     sage -t --long "devel/sage-main/sage/crypto/block_cipher/all.py"
    275              [0.1 s]
    276     sage -t --long "devel/sage-main/sage/crypto/block_cipher/sdes.py"
    277              [3.9 s]
    278     sage -t --long "devel/sage-main/sage/crypto/__init__.py"
    279              [0.0 s]
    280     sage -t --long "devel/sage-main/sage/crypto/stream.py"
    281              [3.3 s]
    282     sage -t --long "devel/sage-main/sage/crypto/classical_cipher.py"
    283              [3.9 s]
    284     sage -t --long "devel/sage-main/sage/crypto/boolean_function.pyx"
    285              [7.2 s]
    286     sage -t --long "devel/sage-main/sage/crypto/lattice.py"
    287              [3.4 s]
    288     sage -t --long "devel/sage-main/sage/crypto/util.py"
    289              [3.3 s]
    290     sage -t --long "devel/sage-main/sage/crypto/cryptosystem.py"
    291              [3.4 s]
    292     sage -t --long "devel/sage-main/sage/crypto/all.py"
    293              [0.1 s]
    294     sage -t --long "devel/sage-main/sage/crypto/mq/__init__.py"
    295              [0.1 s]
    296     sage -t --long "devel/sage-main/sage/crypto/mq/sbox.py"
    297              [3.5 s]
    298     sage -t --long "devel/sage-main/sage/crypto/mq/mpolynomialsystem.py"
    299              [11.8 s]
    300     sage -t --long "devel/sage-main/sage/crypto/mq/sr.py"
    301              [96.8 s]
    302     sage -t --long "devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py"
    303              [2.9 s]
    304     sage -t --long "devel/sage-main/sage/crypto/cipher.py"
    305              [3.2 s]
    306     sage -t --long "devel/sage-main/sage/crypto/classical.py"
    307              [13.6 s]
    308     sage -t --long "devel/sage-main/sage/crypto/public_key/blum_goldwasser.py"
    309              [3.2 s]
    310     sage -t --long "devel/sage-main/sage/crypto/public_key/__init__.py"
    311              [0.1 s]
    312     sage -t --long "devel/sage-main/sage/crypto/public_key/all.py"
    313              [0.1 s]
    314     sage -t --long "devel/sage-main/sage/crypto/stream_cipher.py"
    315              [3.4 s]
    316     sage -t --long "devel/sage-main/sage/crypto/lfsr.py"
    317              [3.0 s]
    318 
    319     ----------------------------------------------------------------------
     283    [jdemeyer@sage sage-5.9]$ ./sage -t --long devel/sage-main/sage/crypto/
     284    Running doctests with ID 2012-07-03-03-48-11-c16721e6.
     285    Doctesting 24 files.
     286    sage -t --long devel/sage-main/sage/crypto/__init__.py
     287        [0 tests, 0.0 s]
     288    sage -t --long devel/sage-main/sage/crypto/all.py
     289        [0 tests, 0.0 s]
     290    sage -t --long devel/sage-main/sage/crypto/boolean_function.pyx
     291        [252 tests, 4.2 s]
     292    sage -t --long devel/sage-main/sage/crypto/cipher.py
     293        [10 tests, 0.0 s]
     294    sage -t --long devel/sage-main/sage/crypto/classical.py
     295        [718 tests, 10.3 s]
     296    sage -t --long devel/sage-main/sage/crypto/classical_cipher.py
     297        [130 tests, 0.5 s]
     298    sage -t --long devel/sage-main/sage/crypto/cryptosystem.py
     299        [82 tests, 0.1 s]
     300    sage -t --long devel/sage-main/sage/crypto/lattice.py
     301        [1 tests, 0.0 s]
     302    sage -t --long devel/sage-main/sage/crypto/lfsr.py
     303        [31 tests, 0.1 s]
     304    sage -t --long devel/sage-main/sage/crypto/stream.py
     305        [17 tests, 0.1 s]
     306    sage -t --long devel/sage-main/sage/crypto/stream_cipher.py
     307        [114 tests, 0.2 s]
     308    sage -t --long devel/sage-main/sage/crypto/util.py
     309        [122 tests, 0.2 s]
     310    sage -t --long devel/sage-main/sage/crypto/block_cipher/__init__.py
     311        [0 tests, 0.0 s]
     312    sage -t --long devel/sage-main/sage/crypto/block_cipher/all.py
     313        [0 tests, 0.0 s]
     314    sage -t --long devel/sage-main/sage/crypto/block_cipher/miniaes.py
     315        [430 tests, 1.1 s]
     316    sage -t --long devel/sage-main/sage/crypto/block_cipher/sdes.py
     317        [290 tests, 0.7 s]
     318    sage -t --long devel/sage-main/sage/crypto/mq/__init__.py
     319        [0 tests, 0.0 s]
     320    sage -t --long devel/sage-main/sage/crypto/mq/mpolynomialsystem.py
     321        [320 tests, 7.5 s]
     322    sage -t --long devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py
     323        [42 tests, 0.1 s]
     324    sage -t --long devel/sage-main/sage/crypto/mq/sbox.py
     325        [124 tests, 0.7 s]
     326    sage -t --long devel/sage-main/sage/crypto/mq/sr.py
     327        [437 tests, 82.4 s]
     328    sage -t --long devel/sage-main/sage/crypto/public_key/__init__.py
     329        [0 tests, 0.0 s]
     330    sage -t --long devel/sage-main/sage/crypto/public_key/all.py
     331        [0 tests, 0.0 s]
     332    sage -t --long devel/sage-main/sage/crypto/public_key/blum_goldwasser.py
     333        [135 tests, 0.2 s]
     334    ------------------------------------------------------------------------
    320335    All tests passed!
    321     Total time for all tests: 174.3 seconds
     336    ------------------------------------------------------------------------
     337    Total time for all tests: 111.8 seconds
     338        cpu time: 106.1 seconds
     339        cumulative wall time: 108.5 seconds
    322340
    323341Notice the time difference between the first set of tests and the
    324342second set, which uses the optional argument ``--long``. Many tests in the
    325343Sage library are flagged with ``# long time`` because these are known to
    326344take a long time to run through. Without using the optional ``--long``
    327 argument, the module ``sage/crypto/mq/sr.py`` took about ten
    328 seconds. With this optional argument, it required 97 seconds to run
     345argument, the module ``sage/crypto/mq/sr.py`` took about five
     346seconds. With this optional argument, it required 82 seconds to run
    329347through all tests in that module. Here is a snippet of a function in
    330348the module ``sage/crypto/mq/sr.py`` with a doctest that has been flagged
    331349as taking a long time::
     
    356374
    357375Now we doctest the same directory in parallel using 4 threads::
    358376
    359     [jdemeyer@sage sage-4.8]$ ./sage -tp 4 devel/sage-main/sage/crypto/
    360     Global iterations: 1
    361     File iterations: 1
    362     Using cached timings to run longest doctests first.
    363     Doctesting 24 files doing 4 jobs in parallel
    364     sage -t  devel/sage-main/sage/crypto/__init__.py
    365              [0.1 s]
    366     sage -t  devel/sage-main/sage/crypto/lattice.py
    367              [3.3 s]
    368     sage -t  devel/sage-main/sage/crypto/stream.py
    369              [3.5 s]
    370     sage -t  devel/sage-main/sage/crypto/classical_cipher.py
    371              [4.0 s]
    372     sage -t  devel/sage-main/sage/crypto/all.py
    373              [0.1 s]
    374     sage -t  devel/sage-main/sage/crypto/util.py
    375              [3.4 s]
    376     sage -t  devel/sage-main/sage/crypto/cryptosystem.py
    377              [3.4 s]
    378     sage -t  devel/sage-main/sage/crypto/boolean_function.pyx
    379              [6.9 s]
    380     sage -t  devel/sage-main/sage/crypto/cipher.py
    381              [3.3 s]
    382     sage -t  devel/sage-main/sage/crypto/block_cipher/__init__.py
    383              [0.1 s]
    384     sage -t  devel/sage-main/sage/crypto/lfsr.py
    385              [3.3 s]
    386     sage -t  devel/sage-main/sage/crypto/stream_cipher.py
    387              [3.4 s]
    388     sage -t  devel/sage-main/sage/crypto/block_cipher/all.py
    389              [0.1 s]
    390     sage -t  devel/sage-main/sage/crypto/mq/__init__.py
    391              [0.1 s]
    392     sage -t  devel/sage-main/sage/crypto/block_cipher/miniaes.py
    393              [4.0 s]
    394     sage -t  devel/sage-main/sage/crypto/block_cipher/sdes.py
    395              [3.6 s]
    396     sage -t  devel/sage-main/sage/crypto/mq/sbox.py
    397              [4.0 s]
    398     sage -t  devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py
    399              [3.2 s]
    400     sage -t  devel/sage-main/sage/crypto/public_key/blum_goldwasser.py
    401              [3.4 s]
    402     sage -t  devel/sage-main/sage/crypto/public_key/__init__.py
    403              [0.1 s]
    404     sage -t  devel/sage-main/sage/crypto/classical.py
    405              [14.3 s]
    406     sage -t  devel/sage-main/sage/crypto/public_key/all.py
    407              [0.1 s]
    408     sage -t  devel/sage-main/sage/crypto/mq/sr.py
    409              [9.3 s]
    410     sage -t  devel/sage-main/sage/crypto/mq/mpolynomialsystem.py
    411              [12.0 s]
    412 
    413     ----------------------------------------------------------------------
     377    [jdemeyer@sage sage-5.9]$ ./sage -tp 4 devel/sage-main/sage/crypto/
     378    Running doctests with ID 2012-07-07-00-11-55-9b17765e.
     379    Sorting sources by runtime so that slower doctests are run first....
     380    Doctesting 24 files using 4 threads.
     381    sage -t devel/sage-main/sage/crypto/boolean_function.pyx
     382        [252 tests, 3.8 s]
     383    sage -t devel/sage-main/sage/crypto/block_cipher/miniaes.py
     384        [429 tests, 1.1 s]
     385    sage -t devel/sage-main/sage/crypto/mq/sr.py
     386        [432 tests, 5.7 s]
     387    sage -t devel/sage-main/sage/crypto/mq/sbox.py
     388        [123 tests, 0.8 s]
     389    sage -t devel/sage-main/sage/crypto/block_cipher/sdes.py
     390        [289 tests, 0.6 s]
     391    sage -t devel/sage-main/sage/crypto/classical_cipher.py
     392        [123 tests, 0.4 s]
     393    sage -t devel/sage-main/sage/crypto/stream_cipher.py
     394        [113 tests, 0.1 s]
     395    sage -t devel/sage-main/sage/crypto/public_key/blum_goldwasser.py
     396        [134 tests, 0.1 s]
     397    sage -t devel/sage-main/sage/crypto/lfsr.py
     398        [30 tests, 0.1 s]
     399    sage -t devel/sage-main/sage/crypto/util.py
     400        [121 tests, 0.1 s]
     401    sage -t devel/sage-main/sage/crypto/cryptosystem.py
     402        [79 tests, 0.0 s]
     403    sage -t devel/sage-main/sage/crypto/stream.py
     404        [12 tests, 0.0 s]
     405    sage -t devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py
     406        [40 tests, 0.0 s]
     407    sage -t devel/sage-main/sage/crypto/cipher.py
     408        [3 tests, 0.0 s]
     409    sage -t devel/sage-main/sage/crypto/lattice.py
     410        [0 tests, 0.0 s]
     411    sage -t devel/sage-main/sage/crypto/block_cipher/__init__.py
     412        [0 tests, 0.0 s]
     413    sage -t devel/sage-main/sage/crypto/all.py
     414        [0 tests, 0.0 s]
     415    sage -t devel/sage-main/sage/crypto/public_key/__init__.py
     416        [0 tests, 0.0 s]
     417    sage -t devel/sage-main/sage/crypto/__init__.py
     418        [0 tests, 0.0 s]
     419    sage -t devel/sage-main/sage/crypto/public_key/all.py
     420        [0 tests, 0.0 s]
     421    sage -t devel/sage-main/sage/crypto/mq/__init__.py
     422        [0 tests, 0.0 s]
     423    sage -t devel/sage-main/sage/crypto/block_cipher/all.py
     424        [0 tests, 0.0 s]
     425    sage -t devel/sage-main/sage/crypto/mq/mpolynomialsystem.py
     426        [318 tests, 8.4 s]
     427    sage -t devel/sage-main/sage/crypto/classical.py
     428        [717 tests, 10.4 s]
     429    ------------------------------------------------------------------------
    414430    All tests passed!
    415     Timings have been updated.
    416     Total time for all tests: 23.7 seconds
    417     [jdemeyer@sage sage-4.8]$ ./sage -tp 4 --long devel/sage-main/sage/crypto/
    418     Global iterations: 1
    419     File iterations: 1
    420     Using long cached timings to run longest doctests first.
    421     Doctesting 24 files doing 4 jobs in parallel
     431    ------------------------------------------------------------------------
     432    Total time for all tests: 12.9 seconds
     433        cpu time: 30.5 seconds
     434        cumulative wall time: 31.7 seconds
     435    [jdemeyer@sage sage-5.9]$ ./sage -tp 4 --long devel/sage-main/sage/crypto/
     436    Running doctests with ID 2012-07-07-00-13-04-d71f3cd4.
     437    Sorting sources by runtime so that slower doctests are run first....
     438    Doctesting 24 files using 4 threads.
     439    sage -t --long devel/sage-main/sage/crypto/boolean_function.pyx
     440        [252 tests, 3.7 s]
     441    sage -t --long devel/sage-main/sage/crypto/block_cipher/miniaes.py
     442        [429 tests, 1.0 s]
     443    sage -t --long devel/sage-main/sage/crypto/mq/sbox.py
     444        [123 tests, 0.8 s]
     445    sage -t --long devel/sage-main/sage/crypto/block_cipher/sdes.py
     446        [289 tests, 0.6 s]
     447    sage -t --long devel/sage-main/sage/crypto/classical_cipher.py
     448        [123 tests, 0.4 s]
     449    sage -t --long devel/sage-main/sage/crypto/util.py
     450        [121 tests, 0.1 s]
     451    sage -t --long devel/sage-main/sage/crypto/stream_cipher.py
     452        [113 tests, 0.1 s]
     453    sage -t --long devel/sage-main/sage/crypto/public_key/blum_goldwasser.py
     454        [134 tests, 0.1 s]
     455    sage -t --long devel/sage-main/sage/crypto/lfsr.py
     456        [30 tests, 0.0 s]
     457    sage -t --long devel/sage-main/sage/crypto/cryptosystem.py
     458        [79 tests, 0.0 s]
     459    sage -t --long devel/sage-main/sage/crypto/stream.py
     460        [12 tests, 0.0 s]
     461    sage -t --long devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py
     462        [40 tests, 0.0 s]
     463    sage -t --long devel/sage-main/sage/crypto/cipher.py
     464        [3 tests, 0.0 s]
     465    sage -t --long devel/sage-main/sage/crypto/lattice.py
     466        [0 tests, 0.0 s]
     467    sage -t --long devel/sage-main/sage/crypto/block_cipher/all.py
     468        [0 tests, 0.0 s]
     469    sage -t --long devel/sage-main/sage/crypto/public_key/__init__.py
     470        [0 tests, 0.0 s]
     471    sage -t --long devel/sage-main/sage/crypto/mq/__init__.py
     472        [0 tests, 0.0 s]
     473    sage -t --long devel/sage-main/sage/crypto/all.py
     474        [0 tests, 0.0 s]
     475    sage -t --long devel/sage-main/sage/crypto/block_cipher/__init__.py
     476        [0 tests, 0.0 s]
    422477    sage -t --long devel/sage-main/sage/crypto/__init__.py
    423              [0.1 s]
    424     sage -t --long devel/sage-main/sage/crypto/stream.py
    425              [3.2 s]
    426     sage -t --long devel/sage-main/sage/crypto/lattice.py
    427              [3.3 s]
    428     sage -t --long devel/sage-main/sage/crypto/classical_cipher.py
    429              [4.1 s]
    430     sage -t --long devel/sage-main/sage/crypto/all.py
    431              [0.1 s]
    432     sage -t --long devel/sage-main/sage/crypto/util.py
    433              [3.1 s]
    434     sage -t --long devel/sage-main/sage/crypto/cryptosystem.py
    435              [3.3 s]
    436     sage -t --long devel/sage-main/sage/crypto/boolean_function.pyx
    437              [7.0 s]
    438     sage -t --long devel/sage-main/sage/crypto/cipher.py
    439              [3.2 s]
    440     sage -t --long devel/sage-main/sage/crypto/block_cipher/__init__.py
    441              [0.1 s]
    442     sage -t --long devel/sage-main/sage/crypto/stream_cipher.py
    443              [3.2 s]
    444     sage -t --long devel/sage-main/sage/crypto/block_cipher/all.py
    445              [0.1 s]
    446     sage -t --long devel/sage-main/sage/crypto/lfsr.py
    447              [3.4 s]
    448     sage -t --long devel/sage-main/sage/crypto/mq/__init__.py
    449              [0.1 s]
    450     sage -t --long devel/sage-main/sage/crypto/block_cipher/miniaes.py
    451              [4.2 s]
    452     sage -t --long devel/sage-main/sage/crypto/block_cipher/sdes.py
    453              [4.0 s]
    454     sage -t --long devel/sage-main/sage/crypto/mq/sbox.py
    455              [3.8 s]
    456     sage -t --long devel/sage-main/sage/crypto/mq/mpolynomialsystemgenerator.py
    457              [3.1 s]
     478        [0 tests, 0.0 s]
     479    sage -t --long devel/sage-main/sage/crypto/public_key/all.py
     480        [0 tests, 0.0 s]
     481    sage -t --long devel/sage-main/sage/crypto/mq/mpolynomialsystem.py
     482        [318 tests, 9.0 s]
    458483    sage -t --long devel/sage-main/sage/crypto/classical.py
    459              [13.8 s]
    460     sage -t --long devel/sage-main/sage/crypto/public_key/__init__.py
    461              [0.0 s]
    462     sage -t --long devel/sage-main/sage/crypto/public_key/all.py
    463              [0.0 s]
    464     sage -t --long devel/sage-main/sage/crypto/public_key/blum_goldwasser.py
    465              [3.1 s]
    466     sage -t --long devel/sage-main/sage/crypto/mq/mpolynomialsystem.py
    467              [11.3 s]
     484        [717 tests, 10.5 s]
    468485    sage -t --long devel/sage-main/sage/crypto/mq/sr.py
    469              [95.4 s]
    470 
    471     ----------------------------------------------------------------------
     486        [434 tests, 88.0 s]
     487    ------------------------------------------------------------------------
    472488    All tests passed!
    473     Total time for all tests: 109.4 seconds
     489    ------------------------------------------------------------------------
     490    Total time for all tests: 90.4 seconds
     491        cpu time: 113.4 seconds
     492        cumulative wall time: 114.5 seconds
    474493
    475494As the number of threads increases, the total testing time
    476495decreases. To minimize confusion, it is also a good idea to explicitly
     
    491510management or patching the main Sage library, a release manager would
    492511parallel test the library using 10 threads with the following command::
    493512
    494     [jdemeyer@sage sage-4.8]$ ./sage -tp 10 -long devel/sage-main/
     513    [jdemeyer@sage sage-5.9]$ ./sage -tp 10 -long devel/sage-main/
    495514
    496515Another way is run ``make ptestlong``, which builds Sage (if necessary),
    497516builds the Sage documentation (if necessary), and then runs parallel
     
    503522
    504523In any case, this will test the Sage library with multiple threads::
    505524
    506     [jdemeyer@sage sage-4.8]$ make ptestlong
     525    [jdemeyer@sage sage-5.9]$ make ptestlong
    507526
    508527Any of the following commands would also doctest the Sage library or
    509528one of its clones::
     
    526545  * ``SAGE_ROOT/devel/sage/doc/fr``
    527546
    528547  Finally, the commands doctest the Sage library. For more details on
    529   these command, see the files ``SAGE_ROOT/Makefile`` and
    530   ``SAGE_ROOT/local/bin/sage-maketest``.
     548  these command, see the file ``SAGE_ROOT/Makefile``.
    531549
    532550* ``make testlong`` --- This command doctests the standard
    533551  documentation:
     
    571589
    572590Then we can doctest it just as with Sage library files::
    573591 
    574     [mvngu@sage build]$ sage-4.8/sage -t my_python_script.py
    575     sage -t  "my_python_script.py"
    576              [1.3 s]
    577 
    578     ----------------------------------------------------------------------
     592    [mvngu@sage sage-5.9]$ ./sage -t my_python_script.py
     593    Running doctests with ID 2012-07-07-00-17-56-d056f7c0.
     594    Doctesting 1 file.
     595    sage -t my_python_script.py
     596        [1 test, 0.0 s]
     597    ------------------------------------------------------------------------
    579598    All tests passed!
    580     Total time for all tests: 1.3 seconds
     599    ------------------------------------------------------------------------
     600    Total time for all tests: 2.2 seconds
     601        cpu time: 0.0 seconds
     602        cumulative wall time: 0.0 seconds
    581603
    582604Doctesting can also be performed on Sage scripts. Say we have a Sage
    583605script called ``my_sage_script.sage`` with the following content::
    584606
    585     [mvngu@sage build]$ cat my_sage_script.sage
     607    [mvngu@sage sage-5.9]$ cat my_sage_script.sage
    586608    def cube(n):
    587609        r"""
    588610        Return the cube of n.
     
    596618
    597619Then we can doctest it just as for Python files::
    598620
    599     [mvngu@sage build]$ sage-4.8/sage --t my_sage_script.sage
    600     sage -t  "my_python_script.sage"
    601              [1.3 s]
    602 
    603     ----------------------------------------------------------------------
     621    [mvngu@sage build]$ sage-5.9/sage -t my_sage_script.sage
     622    Running doctests with ID 2012-07-07-00-20-06-82ee728c.
     623    Doctesting 1 file.
     624    sage -t my_sage_script.sage
     625        [1 test, 0.0 s]
     626    ------------------------------------------------------------------------
    604627    All tests passed!
    605     Total time for all tests: 1.3 seconds
     628    ------------------------------------------------------------------------
     629    Total time for all tests: 2.5 seconds
     630        cpu time: 0.0 seconds
     631        cumulative wall time: 0.0 seconds
    606632
    607633Alternatively, we can preparse it to convert it to a Python script,
    608634and then doctest that::
    609635
    610     [mvngu@sage build]$ sage-4.8/sage --preparse my_sage_script.sage
     636    [mvngu@sage build]$ sage-5.9/sage --preparse my_sage_script.sage
    611637    [mvngu@sage build]$ cat my_sage_script.py
    612638    # This file was *autogenerated* from the file my_sage_script.sage.
    613639    from sage.all_cmdline import *   # import sage library
     
    622648            8
    623649        """
    624650        return n**_sage_const_3
    625     [mvngu@sage build]$ sage-4.8/sage -t my_sage_script.py
    626     sage -t  "my_sage_script.py"
    627              [1.5 s]
     651    [mvngu@sage build]$ sage-5.9/sage -t my_sage_script.py
     652    Running doctests with ID 2012-07-07-00-26-46-2bb00911.
     653    Doctesting 1 file.
     654    sage -t my_sage_script.py
     655        [2 tests, 0.0 s]
     656    ------------------------------------------------------------------------
     657    All tests passed!
     658    ------------------------------------------------------------------------
     659    Total time for all tests: 2.3 seconds
     660        cpu time: 0.0 seconds
     661        cumulative wall time: 0.0 seconds
    628662
    629     ----------------------------------------------------------------------
     663Doctesting from within Sage
     664===========================
     665
     666You can run doctests from within Sage, which can be useful since you
     667don't have to wait for Sage to start.  Use the ``run_doctests``
     668function in the global namespace, passing it either a string or a module::
     669
     670    sage: run_doctests(sage.coding.sd_codes)
     671    Doctesting /Users/roed/sage/sage-5.3/devel/sage/sage/coding/sd_codes.py
     672    Running doctests with ID 2012-07-07-04-32-36-81f3853b.
     673    Doctesting 1 file.
     674    sage -t /Users/roed/sage/sage-5.3/devel/sage/sage/coding/sd_codes.py
     675        [18 tests, 0.3 s]
     676    ------------------------------------------------------------------------
    630677    All tests passed!
    631     Total time for all tests: 1.5 seconds
     678    ------------------------------------------------------------------------
     679    Total time for all tests: 0.4 seconds
     680        cpu time: 0.2 seconds
     681        cumulative wall time: 0.3 seconds
     682
     683.. _section-options:
     684
     685Optional Arguments
     686==================
     687
     688Run long tests
     689--------------
     690
     691Use the ``--long`` flag to run doctests that have been marked with
     692the comment ``# long time``.
     693
     694No doctest should take longer than a second or so, and longer doctests
     695(taking up to 30-60 seconds) should be marked as ``# long time``.
     696These tests are normally skipped in order to reduce the time spent
     697running tests::
     698
     699    [roed@sage sage-5.9]$ sage -t devel/sage/sage/rings/tests.py
     700    Running doctests with ID 2012-06-21-16-00-13-40835825.
     701    Doctesting 1 file.
     702    sage -t tests.py
     703        [18 tests, 1.1 s]
     704    ------------------------------------------------------------------------
     705    All tests passed!
     706    ------------------------------------------------------------------------
     707    Total time for all tests: 2.9 seconds
     708        cpu time: 0.9 seconds
     709        cumulative wall time: 1.1 seconds
     710
     711In order to run the long tests as well, do the following::
     712
     713    [roed@sage sage-5.9]$ sage -t --long devel/sage/sage/rings/tests.py
     714    Running doctests with ID 2012-06-21-16-02-05-d13a9a24.
     715    Doctesting 1 file.
     716    sage -t tests.py
     717        [20 tests, 34.7 s]
     718    ------------------------------------------------------------------------
     719    All tests passed!
     720    ------------------------------------------------------------------------
     721    Total time for all tests: 46.5 seconds
     722        cpu time: 25.2 seconds
     723        cumulative wall time: 34.7 seconds
     724
     725To find tests that take longer than the allowed time use the
     726``--warn-long`` flag.  Without any options it will cause tests to fail
     727if they take longer than 1.0 second::
     728
     729    [roed@sage sage-5.9]$ sage -t --warn-long devel/sage/sage/rings/factorint.pyx
     730    Running doctests with ID 2012-07-14-03-27-03-2c952ac1.
     731    Doctesting 1 file.
     732    sage -t --warn-long devel/sage/sage/rings/factorint.pyx
     733    **********************************************************************
     734    File "devel/sage/sage/rings/factorint.pyx", line 125, in sage.rings.factorint.base_exponent
     735    Failed example:
     736        base_exponent(-4)
     737    Test ran for 4.09 s
     738    **********************************************************************
     739    File "devel/sage/sage/rings/factorint.pyx", line 153, in sage.rings.factorint.factor_aurifeuillian
     740    Failed example:
     741        fa(2^6+1)
     742    Test ran for 2.22 s
     743    **********************************************************************
     744    File "devel/sage/sage/rings/factorint.pyx", line 155, in sage.rings.factorint.factor_aurifeuillian
     745    Failed example:
     746        fa(2^58+1)
     747    Test ran for 2.22 s
     748    **********************************************************************
     749    File "devel/sage/sage/rings/factorint.pyx", line 163, in sage.rings.factorint.factor_aurifeuillian
     750    Failed example:
     751        fa(2^4+1)
     752    Test ran for 2.25 s
     753    **********************************************************************
     754    2 items had failures:
     755       1 of   6 in sage.rings.factorint.base_exponent
     756       3 of   8 in sage.rings.factorint.factor_aurifeuillian
     757        [25 tests, 4 failures, 10.9 s]
     758    ------------------------------------------------------------------------
     759    sage -t --warn-long devel/sage/sage/rings/factorint.pyx # 4 doctests failed
     760    ------------------------------------------------------------------------
     761    Total time for all tests: 16.1 seconds
     762        cpu time: 9.7 seconds
     763        cumulative wall time: 10.9 seconds
     764
     765You can also pass in an explicit amount of time::
     766
     767    [roed@sage sage-5.9]$ sage -t --long --warn-long 2.0 devel/sage/sage/rings/tests.py
     768    Running doctests with ID 2012-07-14-03-30-13-c9164c9d.
     769    Doctesting 1 file.
     770    sage -t --long --warn-long 2.0 tests.py
     771    **********************************************************************
     772    File "tests.py", line 240, in sage.rings.tests.test_random_elements
     773    Failed example:
     774        sage.rings.tests.test_random_elements(trials=1000)  # long time (5 seconds)
     775    Test ran for 13.36 s
     776    **********************************************************************
     777    File "tests.py", line 283, in sage.rings.tests.test_random_arith
     778    Failed example:
     779        sage.rings.tests.test_random_arith(trials=1000)   # long time (5 seconds?)
     780    Test ran for 12.42 s
     781    **********************************************************************
     782    2 items had failures:
     783       1 of   4 in sage.rings.tests.test_random_arith
     784       1 of   4 in sage.rings.tests.test_random_elements
     785        [20 tests, 2 failures, 26.3 s]
     786    ------------------------------------------------------------------------
     787    sage -t --long --warn-long 2.0 tests.py # 2 doctests failed
     788    ------------------------------------------------------------------------
     789    Total time for all tests: 27.6 seconds
     790        cpu time: 24.8 seconds
     791        cumulative wall time: 26.3 seconds
     792
     793Run optional tests
     794------------------
     795
     796You can run tests that require optional packages by using the
     797``--optional`` flag.  Obviously, you need to have installed the
     798necessary optional packages in order for these tests to succeed.  See
     799http://www.sagemath.org/packages/optional/ in order to download
     800optional packages.
     801
     802By default, Sage only runs doctests that are not marked with the ``optional`` tag.  This is equivalent to running ::
     803
     804    [roed@sage sage-5.9]$ sage -t --optional=sage devel/sage/sage/rings/real_mpfr.pyx
     805    Running doctests with ID 2012-06-21-16-18-30-a368a200.
     806    Doctesting 1 file.
     807    sage -t devel/sage/sage/rings/real_mpfr.pyx
     808        [819 tests, 7.0 s]
     809    ------------------------------------------------------------------------
     810    All tests passed!
     811    ------------------------------------------------------------------------
     812    Total time for all tests: 8.4 seconds
     813        cpu time: 4.1 seconds
     814        cumulative wall time: 7.0 seconds
     815
     816If you want to also run tests that require magma, you can do the following::
     817
     818    [roed@sage sage-5.9]$ sage -t --optional=sage,magma devel/sage/sage/rings/real_mpfr.pyx
     819    Running doctests with ID 2012-06-21-16-18-30-a00a7319
     820    Doctesting 1 file.
     821    sage -t devel/sage/sage/rings/real_mpfr.pyx
     822        [823 tests, 8.4 s]
     823    ------------------------------------------------------------------------
     824    All tests passed!
     825    ------------------------------------------------------------------------
     826    Total time for all tests: 9.6 seconds
     827        cpu time: 4.0 seconds
     828        cumulative wall time: 8.4 seconds
     829
     830In order to just run the tests that are marked as requiring magma, omit ``sage``::
     831
     832    [roed@sage sage-5.9]$ sage -t --optional=magma devel/sage/sage/rings/real_mpfr.pyx
     833    Running doctests with ID 2012-06-21-16-18-33-a2bc1fdf
     834    Doctesting 1 file.
     835    sage -t devel/sage/sage/rings/real_mpfr.pyx
     836        [4 tests, 2.0 s]
     837    ------------------------------------------------------------------------
     838    All tests passed!
     839    ------------------------------------------------------------------------
     840    Total time for all tests: 3.2 seconds
     841        cpu time: 0.1 seconds
     842        cumulative wall time: 2.0 seconds
     843
     844To run all tests, regardless of whether they are marked optional, pass ``all`` as the ``optional`` tag::
     845
     846    [roed@sage sage-5.9]$ sage -t --optional=all devel/sage/sage/rings/real_mpfr.pyx
     847    Running doctests with ID 2012-06-21-16-31-18-8c097f55
     848    Doctesting 1 file.
     849    sage -t devel/sage/sage/rings/real_mpfr.pyx
     850        [865 tests, 11.2 s]
     851    ------------------------------------------------------------------------
     852    All tests passed!
     853    ------------------------------------------------------------------------
     854    Total time for all tests: 12.8 seconds
     855        cpu time: 4.7 seconds
     856        cumulative wall time: 11.2 seconds
     857
     858Running tests in parallel
     859-------------------------
     860
     861If you're testing many files, you can get big speedups by using more
     862than one thread.  To run doctests in parallel use the ``--nthreads``
     863flag (``-p`` is a shortened version).  Pass in the number of threads
     864you would like to use (by default Sage just uses 1)::
     865
     866    [roed@sage sage-5.9]$ sage -tp 2 devel/sage/sage/doctest/
     867    Running doctests with ID 2012-06-22-19-09-25-a3afdb8c.
     868    Sorting sources by runtime so that slower doctests are run first....
     869    Doctesting 8 files using 2 threads.
     870    sage -t devel/sage/sage/doctest/control.py
     871        [114 tests, 4.6 s]
     872    sage -t devel/sage/sage/doctest/util.py
     873        [114 tests, 0.6 s]
     874    sage -t devel/sage/sage/doctest/parsing.py
     875        [187 tests, 0.5 s]
     876    sage -t devel/sage/sage/doctest/sources.py
     877        [128 tests, 0.1 s]
     878    sage -t devel/sage/sage/doctest/reporting.py
     879        [53 tests, 0.1 s]
     880    sage -t devel/sage/sage/doctest/all.py
     881        [0 tests, 0.0 s]
     882    sage -t devel/sage/sage/doctest/__init__.py
     883        [0 tests, 0.0 s]
     884    sage -t devel/sage/sage/doctest/forker.py
     885        [322 tests, 15.5 s]
     886    ------------------------------------------------------------------------
     887    All tests passed!
     888    ------------------------------------------------------------------------
     889    Total time for all tests: 17.0 seconds
     890        cpu time: 4.2 seconds
     891        cumulative wall time: 21.5 seconds
     892
     893Doctesting all of Sage
     894----------------------
     895
     896To doctest the whole Sage library use the ``--all`` flag (``-a`` for
     897short).  In addition to testing the code in Sage's Python and Cython
     898files, this command will run the tests defined in Sage's documentation
     899as well as testing the Sage notebook::
     900
     901    [roed@sage sage-5.9]$ sage -t -a
     902    Running doctests with ID 2012-06-22-19-10-27-e26fce6d.
     903    Doctesting entire Sage library.
     904    Sorting sources by runtime so that slower doctests are run first....
     905    Doctesting 2020 files.
     906    sage -t /Users/roed/sage/sage-5.3/devel/sage/sage/plot/plot.py
     907        [304 tests, 69.0 s]
     908    ...
     909   
     910
     911If you want to just run the notebook tests, use the ``--sagenb`` flag instead.
     912
     913Debugging tools
     914---------------
     915
     916Sometimes doctests fail (that's why we run them after all).  There are
     917various flags to help when something goes wrong.  If a doctest
     918produces a Python error, then normally tests continue after reporting
     919that an error occurred.  If you use the flag ``--debug`` (``-d`` for
     920short) then you will drop into an interactive Python debugger whenever
     921a Python exception occurs.  As an example, I modified
     922:mod:`sage.schemes.elliptic_curves.constructor` to produce an error::
     923
     924    [roed@sage sage-5.9]$ sage -t --debug devel/sage/sage/schemes/elliptic_curves/constructor.py
     925    Running doctests with ID 2012-06-23-12-09-04-b6352629.
     926    Doctesting 1 file.
     927    **********************************************************************
     928    File "sage.schemes.elliptic_curves.constructor", line 4, in sage.schemes.elliptic_curves.constructor
     929    Failed example:
     930        EllipticCurve([0,0])
     931    Exception raised:
     932        Traceback (most recent call last):
     933          File "/Users/roed/sage/sage-5.3/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 573, in _run
     934            self.execute(example, compiled, test.globs)
     935          File "/Users/roed/sage/sage-5.3/local/lib/python2.7/site-packages/sage/doctest/forker.py", line 835, in execute
     936            exec compiled in globs
     937          File "<doctest sage.schemes.elliptic_curves.constructor[0]>", line 1, in <module>
     938            EllipticCurve([Integer(0),Integer(0)])
     939          File "/Users/roed/sage/sage-5.3/local/lib/python2.7/site-packages/sage/schemes/elliptic_curves/constructor.py", line 346, in EllipticCurve
     940            return ell_rational_field.EllipticCurve_rational_field(x, y)
     941          File "/Users/roed/sage/sage-5.3/local/lib/python2.7/site-packages/sage/schemes/elliptic_curves/ell_rational_field.py", line 216, in __init__
     942            EllipticCurve_number_field.__init__(self, Q, ainvs)
     943          File "/Users/roed/sage/sage-5.3/local/lib/python2.7/site-packages/sage/schemes/elliptic_curves/ell_number_field.py", line 159, in __init__
     944            EllipticCurve_field.__init__(self, [field(x) for x in ainvs])
     945          File "/Users/roed/sage/sage-5.3/local/lib/python2.7/site-packages/sage/schemes/elliptic_curves/ell_generic.py", line 156, in __init__
     946            "Invariants %s define a singular curve."%ainvs
     947        ArithmeticError: Invariants [0, 0, 0, 0, 0] define a singular curve.
     948    > /Users/roed/sage/sage-5.3/local/lib/python2.7/site-packages/sage/schemes/elliptic_curves/ell_generic.py(156)__init__()
     949    -> "Invariants %s define a singular curve."%ainvs
     950    (Pdb) l
     951    151                 if len(ainvs) == 2:
     952    152                     ainvs = [K(0),K(0),K(0)] + ainvs
     953    153                 self.__ainvs = tuple(ainvs)
     954    154                 if self.discriminant() == 0:
     955    155                     raise ArithmeticError, \
     956    156  ->                       "Invariants %s define a singular curve."%ainvs
     957    157                 PP = projective_space.ProjectiveSpace(2, K, names='xyz');
     958    158                 x, y, z = PP.coordinate_ring().gens()
     959    159                 a1, a2, a3, a4, a6 = ainvs
     960    160                 f = y**2*z + (a1*x + a3*z)*y*z \
     961    161                     - (x**3 + a2*x**2*z + a4*x*z**2 + a6*z**3)
     962    (Pdb) p ainvs
     963    [0, 0, 0, 0, 0]
     964    (Pdb) quit
     965    **********************************************************************
     966    1 items had failures:
     967       1 of   1 in sage.schemes.elliptic_curves.constructor
     968    ***Test Failed*** 1 failures.
     969    sage -t devel/sage/sage/schemes/elliptic_curves/constructor.py
     970        [64 tests, 89.2 s]
     971    ------------------------------------------------------------------------
     972    sage -t devel/sage/sage/schemes/elliptic_curves/constructor.py # 1 doctest failed
     973    ------------------------------------------------------------------------
     974    Total time for all tests: 90.4 seconds
     975        cpu time: 4.5 seconds
     976        cumulative wall time: 89.2 seconds
     977
     978Sometimes an error might be so severe that it causes Sage to segfault
     979or hang.  In such a situation you have a number of options.  The
     980doctest framework will print out the output so far, so that at least
     981you know what test caused the problem (if you want this output to
     982appear in real time use the ``--verbose`` flag).  To have doctests run
     983under the control of gdb, use the ``--gdb`` flag::
     984
     985    [roed@sage sage-5.9]$ sage -t --gdb devel/sage/sage/schemes/elliptic_curves/constructor.py
     986    gdb -x /home/roed/sage-5.9.b5/local/bin/sage-gdb-commands --args python /home/roed/sage-5.9.b5/local/bin/sage-runtests --serial --nthreads 1 --timeout 1048576 --optional sage --stats_path /home/roed/.sage/timings2.json devel/sage/sage/schemes/elliptic_curves/constructor.py
     987    GNU gdb 6.8-debian
     988    Copyright (C) 2008 Free Software Foundation, Inc.
     989    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
     990    This is free software: you are free to change and redistribute it.
     991    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
     992    and "show warranty" for details.
     993    This GDB was configured as "x86_64-linux-gnu"...
     994    [Thread debugging using libthread_db enabled]
     995    [New Thread 0x7f10f85566e0 (LWP 6534)]
     996    Running doctests with ID 2012-07-07-00-43-36-b1b735e7.
     997    Doctesting 1 file.
     998    sage -t devel/sage/sage/schemes/elliptic_curves/constructor.py
     999        [67 tests, 5.8 s]
     1000    ------------------------------------------------------------------------
     1001    All tests passed!
     1002    ------------------------------------------------------------------------
     1003    Total time for all tests: 15.7 seconds
     1004        cpu time: 4.4 seconds
     1005        cumulative wall time: 5.8 seconds
     1006
     1007    Program exited normally.
     1008    (gdb) quit
     1009
     1010
     1011Sage also includes valgrind, and you can run doctests under various
     1012valgrind tools to track down memory issues: the relevant flags are
     1013``--valgrind`` (or ``--memcheck``), ``--massif``, ``--cachegrind`` and
     1014``--omega``.  See http://wiki.sagemath.org/ValgrindingSage for more details.
     1015
     1016Once you're done fixing whatever problems where revealed by the
     1017doctests, you can rerun just those files that failed their most recent
     1018test by using the ``--failed`` flag (``-f`` for short)::
     1019
     1020    [roed@sage sage-5.9]$ sage -t -fa
     1021    Running doctests with ID 2012-07-07-00-45-35-d8b5a408.
     1022    Doctesting entire Sage library.
     1023    Only doctesting files that failed last test.
     1024    No files to doctest
     1025
     1026Other options
     1027-------------
     1028
     1029There are various other options that change the behavior of Sage's doctesting code.
     1030
     1031Show only first failure
     1032^^^^^^^^^^^^^^^^^^^^^^^
     1033
     1034The first failure in a file often causes a cascade of others, as
     1035NameErrors arise from variables that weren't defined and tests fail
     1036because old values of variables are used.  To only see the first
     1037failure in each doctest block use the ``--initial`` flag (``-i`` for
     1038short).
     1039
     1040Running tests with iterations
     1041^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     1042
     1043Sometimes tests fail intermittently.  There are two options that allow
     1044you to run tests repeatedly in an attempt to search for Heisenbugs.
     1045The flag ``--global-iterations`` takes an integer and runs the whole
     1046set of tests that many times serially::
     1047
     1048    [roed@sage sage-5.9]$ sage -t --global-iterations 2 devel/sage-main/sage/sandpiles
     1049    Running doctests with ID 2012-07-07-00-59-28-e7048ad9.
     1050    Doctesting 3 files (2 global iterations).
     1051    sage -t devel/sage-main/sage/sandpiles/__init__.py
     1052        [0 tests, 0.0 s]
     1053    sage -t devel/sage-main/sage/sandpiles/all.py
     1054        [0 tests, 0.0 s]
     1055    sage -t devel/sage-main/sage/sandpiles/sandpile.py
     1056        [711 tests, 14.7 s]
     1057    ------------------------------------------------------------------------
     1058    All tests passed!
     1059    ------------------------------------------------------------------------
     1060    Total time for all tests: 17.6 seconds
     1061        cpu time: 13.2 seconds
     1062        cumulative wall time: 14.7 seconds
     1063    sage -t devel/sage-main/sage/sandpiles/__init__.py
     1064        [0 tests, 0.0 s]
     1065    sage -t devel/sage-main/sage/sandpiles/all.py
     1066        [0 tests, 0.0 s]
     1067    sage -t devel/sage-main/sage/sandpiles/sandpile.py
     1068        [711 tests, 13.8 s]
     1069    ------------------------------------------------------------------------
     1070    All tests passed!
     1071    ------------------------------------------------------------------------
     1072    Total time for all tests: 14.3 seconds
     1073        cpu time: 26.4 seconds
     1074        cumulative wall time: 28.5 seconds
     1075
     1076You can also iterate in a different order: the ``--file-iterations``
     1077flag runs the tests in each file ``N`` times before proceeding::
     1078
     1079    [roed@sage sage-5.9]$ sage -t --file-iterations 2 devel/sage-main/sage/sandpiles
     1080    Running doctests with ID 2012-07-07-01-01-43-8f954206.
     1081    Doctesting 3 files (2 file iterations).
     1082    sage -t devel/sage-main/sage/sandpiles/__init__.py
     1083        [0 tests, 0.0 s]
     1084    sage -t devel/sage-main/sage/sandpiles/all.py
     1085        [0 tests, 0.0 s]
     1086    sage -t devel/sage-main/sage/sandpiles/sandpile.py
     1087        [1422 tests, 13.3 s]
     1088    ------------------------------------------------------------------------
     1089    All tests passed!
     1090    ------------------------------------------------------------------------
     1091    Total time for all tests: 29.6 seconds
     1092        cpu time: 12.7 seconds
     1093        cumulative wall time: 13.3 seconds
     1094
     1095
     1096Note that the reported results are the average time for all tests in
     1097that file to finish.  If a failure in a file occurs, then the failure
     1098is reported and testing proceeds with the next file.
     1099
     1100Using a different timeout
     1101^^^^^^^^^^^^^^^^^^^^^^^^^
     1102
     1103On a slow machine the default timeout of 5 minutes may not be enough
     1104for the slowest files.  Use the ``--timeout`` flag (``-T`` for short)
     1105to set it to something else::
     1106
     1107    [roed@sage sage-5.9]$ sage -tp 2 --all --timeout 1
     1108    Running doctests with ID 2012-07-07-01-09-37-deb1ab83.
     1109    Doctesting entire Sage library.
     1110    Sorting sources by runtime so that slower doctests are run first....
     1111    Doctesting 2067 files using 2 threads.
     1112    sage -t devel/sage-main/sage/schemes/elliptic_curves/ell_rational_field.py
     1113        Timed out!
     1114    ...
     1115
     1116Using absolute paths
     1117^^^^^^^^^^^^^^^^^^^^
     1118
     1119By default filenames are printed using relative paths.  To use
     1120absolute paths instead pass in the ``--abspath`` flag::
     1121
     1122    [roed@sage sage-5.9]$ sage -t --abspath devel/sage-main/sage/doctest/control.py
     1123    Running doctests with ID 2012-07-07-01-13-03-a023e212.
     1124    Doctesting 1 file.
     1125    sage -t /home/roed/sage-5.9/devel/sage-main/sage/doctest/control.py
     1126        [133 tests, 4.7 s]
     1127    ------------------------------------------------------------------------
     1128    All tests passed!
     1129    ------------------------------------------------------------------------
     1130    Total time for all tests: 7.1 seconds
     1131        cpu time: 0.2 seconds
     1132        cumulative wall time: 4.7 seconds
     1133
     1134Testing changed files
     1135^^^^^^^^^^^^^^^^^^^^^
     1136
     1137If you're working on some files in the Sage library it can be
     1138convenient to test only the files that have changed.  To do so use the
     1139``--new`` flag, which tests files that have been modified or added
     1140since the last Mercurial commit (or qrefresh)::
     1141
     1142    [roed@sage sage-5.9]$ sage -t --new
     1143    Running doctests with ID 2012-07-07-01-15-52-645620ee.
     1144    Doctesting files changed since last HG commit.
     1145    Doctesting 1 file.
     1146    sage -t devel/sage/sage/doctest/control.py
     1147        [133 tests, 3.7 s]
     1148    ------------------------------------------------------------------------
     1149    All tests passed!
     1150    ------------------------------------------------------------------------
     1151    Total time for all tests: 3.8 seconds
     1152        cpu time: 0.1 seconds
     1153        cumulative wall time: 3.7 seconds
     1154
     1155Running tests in a random order
     1156^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
     1157
     1158By default, tests are run in the order in which they appear in the
     1159file.  To run tests in a random order (which can reveal subtle bugs),
     1160use the ``--randorder`` flag and pass in a random seed::
     1161
     1162    [roed@sage sage-5.9]$ sage -t --new --randorder 127
     1163    Running doctests with ID 2012-07-07-01-19-06-97c8484e.
     1164    Doctesting files changed since last HG commit.
     1165    Doctesting 1 file.
     1166    sage -t devel/sage/sage/doctest/control.py
     1167        [133 tests, 3.6 s]
     1168    ------------------------------------------------------------------------
     1169    All tests passed!
     1170    ------------------------------------------------------------------------
     1171    Total time for all tests: 3.7 seconds
     1172        cpu time: 0.2 seconds
     1173        cumulative wall time: 3.6 seconds
     1174
     1175Note that even with this option, the tests within a given doctest block are still run in order.
     1176
     1177Testing external files
     1178^^^^^^^^^^^^^^^^^^^^^^
     1179
     1180When testing a file that's not part of the Sage library, the testing
     1181code loads the globals from that file into the namespace before
     1182running tests.  To model the behavior used on the Sage library instead
     1183(where imports must be explicitly specified), use the ``--force-lib``
     1184flag.
     1185
     1186Auxilliary files
     1187^^^^^^^^^^^^^^^^
     1188
     1189To specify a logfile (rather than use the default which is created for
     1190``sage -t --all``), use the ``--logfile`` flag::
     1191
     1192    [roed@sage sage-5.9]$ sage -t --logfile test1.log devel/sage-main/sage/doctest/control.py
     1193    Running doctests with ID 2012-07-07-01-25-49-e7c0e52d.
     1194    Doctesting 1 file.
     1195    sage -t devel/sage-main/sage/doctest/control.py
     1196        [133 tests, 4.3 s]
     1197    ------------------------------------------------------------------------
     1198    All tests passed!
     1199    ------------------------------------------------------------------------
     1200    Total time for all tests: 6.7 seconds
     1201        cpu time: 0.1 seconds
     1202        cumulative wall time: 4.3 seconds
     1203    [roed@sage sage-5.9]$ cat test1.log
     1204    Running doctests with ID 2012-07-07-01-25-49-e7c0e52d.
     1205    Doctesting 1 file.
     1206    sage -t devel/sage-main/sage/doctest/control.py
     1207        [133 tests, 4.3 s]
     1208    ------------------------------------------------------------------------
     1209    All tests passed!
     1210    ------------------------------------------------------------------------
     1211    Total time for all tests: 6.7 seconds
     1212        cpu time: 0.1 seconds
     1213        cumulative wall time: 4.3 seconds
     1214
     1215
     1216To give a json file storing the timings for each file, use the
     1217``--stats_path`` flag.  These statistics are used in sorting files so
     1218that slower tests are run first (and thus multiple processes are
     1219utilized most efficiently)::
     1220
     1221    [roed@sage sage-5.9]$ sage -tp 2 --stats-path ~/.sage/timings2.json --all
     1222    Running doctests with ID 2012-07-07-01-28-34-2df4251d.
     1223    Doctesting entire Sage library.
     1224    Sorting sources by runtime so that slower doctests are run first....
     1225    Doctesting 2067 files using 2 threads.
     1226    ...