Update doctesting framework

[Robert Bradshaw]

There are several improvements that would be good to make, including but not limited to:

1. Forking rather than starting up a new Sage each time.
2. Customizable comparison rather than mutating doctests into new doctests (e.g. followup to #10952).
3. Eliminate need to write temporary files for every doctested file.
4. Constant headaches caused by the difference in doctesting library vs. non-library files.
5. The ability to drop into a debugger if a doctest raises an exception.

Robert's code is at

​http://code.google.com/p/sagemath-timer/

and does most of the above (plus more which has been moved to followup tickets #12720 and #12722)

Apply attachment:12415_stderr_vs_51b5.patch, attachment:12415_doctest_fixes.patch​, attachment:12415_framework.patch​

Apply attachment:12415_script.patch​ to the scripts repo

Apply attachment:12415_sagenb_fixes.patch to the sagenb repo (fixes doctest errors)

Apply attachment:12415_spkg_bin_sage.patch​ to $SAGE_ROOT/spkg/bin/sage (there's no repo here so you need to use patch)

[David Roe]

I'm working on this. E-mail me if you want to collaborate.

[Robert Bradshaw]

12415_2.patch seems to contain a lot of tempfile changes, could you split out the doctest changes into a separate patch (or are these affecting doctests?)

[R. Andrew Ohana]

apparently even I forget my trac username is not what it should be

[David Roe]

I've moved the patches dealing with temporary files to #13147.

[David Roe]

Ready for review!

There are still a few known issues:

• valgrinding under sage -t doesn't work. I haven't used valgrind much and don't know how to resolve this.
• The fix at #13147 probably needs work: I'm getting a failure in sage.rings.polynomial.multi_polynomial_libsingular to do with garbage collection. I'm working on it.
• I'm in the process of updating the developer guide.

But I wanted to get some other people involved in reviewing.

[John Palmieri]

This may be a silly question, but how do I use the new framework? Don't we need changes to the scripts repo, too? (If I just run sage -t FILES, I don't see the various messages, like Doctesting ..., that should be printed by run_doctests are not printed, so sage -t is not yet using the new framework.)

One of the patches also didn't apply cleanly to 5.1.beta5:

hg qimport -P ~/Downloads/12415_stderr.patch 
adding 12415_stderr.patch to series file
applying 12415_stderr.patch
patching file sage/lfunctions/lcalc.py
Hunk #1 FAILED at 226
1 out of 2 hunks FAILED -- saving rejects to file sage/lfunctions/lcalc.py.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
errors during apply, please fix and refresh 12415_stderr.patch

[David Roe]

Did you apply the new dependencies (#13145, #13146, #13147)? There are some more patches to other repositories: I'll update the ticket description.