bugs in doctesting script for examples with tolerance

[mstreng]

1. When testing the file

   """
   EXAMPLES:
   
   The name blah is undefined::
   
       sage: blah+blah
       blahblah
   
   Here's a doctest that is out of tolerance::
   
       sage: 100+10000000000 # abs tol 0.1
       10000000000
       sage: 1+1
       2
   """
   

I get

File "/Users/marcostreng/tmp/test.sage", line 6:
    sage: blah+blah
Out of tolerance 10000000000.0 vs 10000000100.0

So the error of "blah is undefined" is not printed, while the "out of tolerance" is printed for the wrong test.

2. Also, there is a bug when a test with tolerance is the last test:

   """
   EXAMPLES:
   
   Doctests with tolerance cannot be followed by empty lines::
   
       sage: 100+10000000000 # rel tol 0.1
       10000000000
   """
   

yields

  File "/Users/marcostreng/.sage//tmp/test_71331.py", line 58
    ... ''', res, 0.1, 're
    ^
SyntaxError: invalid syntax

3. The following is documented to work, but actually doesn't work:

   """
   sage: 1e16     # relative tol 1e-10
   10^16
   
   """
   

Testing this gives

sage -t  "/home/jdemeyer/doctest/tolerance_fail.py"
**********************************************************************
File "/home/jdemeyer/doctest/tolerance_fail.py", line 2:
    sage: 1e16     # relative tol 1e-10
Expected '
    10^16
    ' got '1.00000000000000e16'
**********************************************************************

[jdemeyer]

A patch at #13899 documents the third bug in the ticket description.

[jdemeyer]

Issues 1 and 2 are fixed by #12415. Issue 3 might be "wontfix" but then the documentation needs to be changed.

[jdemeyer]

Testing issue 3 with #12415 gives

sage -t /home/jdemeyer/doctest/tolerance_fail.py
**********************************************************************
File "/home/jdemeyer/doctest/tolerance_fail.py", line 2, in tolerance_fail
Failed example:
    1e16     # relative tol 1e-10
Expected:
    10^16
Got:
    1.00000000000000e16
**********************************************************************

[mstreng]

Replying to jdemeyer:

Testing issue 3 with #12415 gives

Yes, and that is what the last block of changes of  trac_13899-doctest-rebase.patch is for.

[robertwb]

Replying to jdemeyer:

Testing issue 3 with #12415 gives

sage -t /home/jdemeyer/doctest/tolerance_fail.py
**********************************************************************
File "/home/jdemeyer/doctest/tolerance_fail.py", line 2, in tolerance_fail
Failed example:
    1e16     # relative tol 1e-10
Expected:
    10^16
Got:
    1.00000000000000e16
**********************************************************************

I consider this a feature, not a bug. If we're going to allow arithmetic expressions, where do we draw the line?

sage: 1e16    # rel tol 1e-10
100000000^2
sage: 1e16    # rel tol 1e-10
4e15 + 6e15
sage: 1e16    # rel tol 1e-10
solve(1e16 - x, x)[0].rhs()

There is also the advantage of being able to write

sage: "%g is less than %g" % (sqrt(2), pi)   # rel tol .01
'1.41 is less than 3.14'

which is possible because we do a match on floating point literals.

[mstreng]

Replying to robertwb:

I consider this a feature, not a bug.

+1 (for the reasons Robert mentions)

Jeroen, could appropriately change the documentation in  trac_13899-doctest-rebase.patch?

[jdemeyer]

I propose the following: we add a patch to this ticket to fix the documentation concerning issue 3 and close the first two issues as fixed by #12415.

[mstreng]

Replying to jdemeyer:

I propose the following: we add a patch to this ticket to fix the documentation concerning issue 3 and close the first two issues as fixed by #12415.

Sounds good to me.

[jdemeyer]

Patch to clarify what tolerance testing does. Needs review.

[jdemeyer]

Could anybody please review this ticket?

[kcrisman]

Is this patch "backwards"?

[robertwb]

No, the patch is not backwards. The description looks good to me.

[kcrisman]

No, the patch is not backwards. The description looks good to me.

Thanks, I understand now; I was getting confused with #13899.