Opened 12 years ago

Last modified 7 years ago

#10458 closed defect

Doctest framework fails to parse multiline input pasted from sage interactive prompt — at Version 5

Reported by: Keshav Kini Owned by: Minh Van Nguyen
Priority: major Milestone: sage-duplicate/invalid/wontfix
Component: doctest coverage Keywords: doctest, continuation, multiline input, interactive prompt
Cc: Merged in:
Authors: Keshav Kini Reviewers:
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description (last modified by Leif Leonhardy)

See this sage-devel thread for some discussion about the changes proposed by this ticket.

After some confusion trying to get a docstring to pass doctesting, I think input line continuations are "not working", in that lines directly pasted from the sage interpreter prompt do not pass doctests when single inputs span multiple lines. Pasted below is some terminal output (analysis continued afterwards):

keshav@esterhazy /opt/sage/devel $ cat test.rst 
EXAMPLE::

    sage: [1,2,3]
    [1, 2, 3]
    sage: [1,
    ....: 2,3
    ....: ]
    [1, 2, 3]
    sage: x=3;\
    ....: x
    3

keshav@esterhazy /opt/sage/devel $ sage -t -verbose test.rst 
sage -t -verbose "4.6/devel/test.rst"                       
Traceback (most recent call last):
  File "/home/keshav/.sage//tmp/.doctest_test.py", line 60, in <module>
    runner=runner)
  File "/opt/sage/local/bin/sagedoctest.py", line 54, in testmod_returning_runner
    runner=runner)
  File "/opt/sage/local/bin/ncadoctest.py", line 1819, in testmod_returning_runner
    for test in finder.find(m, name, globs=globs, extraglobs=extraglobs):
  File "/opt/sage/local/bin/ncadoctest.py", line 839, in find
    self._find(tests, obj, name, module, source_lines, globs, {})
  File "/opt/sage/local/bin/ncadoctest.py", line 893, in _find
    globs, seen)
  File "/opt/sage/local/bin/ncadoctest.py", line 881, in _find
    test = self._get_test(obj, name, module, globs, source_lines)
  File "/opt/sage/local/bin/ncadoctest.py", line 965, in _get_test
    filename, lineno)
  File "/opt/sage/local/bin/ncadoctest.py", line 594, in get_doctest
    return DocTest(self.get_examples(string, name), globs,
  File "/opt/sage/local/bin/ncadoctest.py", line 608, in get_examples
    return [x for x in self.parse(string, name)
  File "/opt/sage/local/bin/ncadoctest.py", line 570, in parse
    self._parse_example(m, name, lineno)
  File "/opt/sage/local/bin/ncadoctest.py", line 628, in _parse_example
    self._check_prompt_blank(source_lines, indent, name, lineno)
  File "/opt/sage/local/bin/ncadoctest.py", line 715, in _check_prompt_blank
    line[indent:indent+3], line))
ValueError: line 10 of the docstring for __main__.example_0 lacks blank after ...: '    ....: Integer(2),Integer(3)'
	 [3.1 s]
 
----------------------------------------------------------------------
The following tests failed:


	sage -t -verbose "4.6/devel/test.rst" # Exception from doctest framework
Total time for all tests: 3.1 seconds
keshav@esterhazy /opt/sage/devel $ sed "s/\.\.\.:/\.\./" test.rst > test2.rst
keshav@esterhazy /opt/sage/devel $ sage -t -verbose test2.rst 
sage -t -verbose "4.6/devel/test2.rst"                      
Trying:
    set_random_seed(0L)
Expecting nothing
ok
Trying:
    change_warning_output(sys.stdout)
Expecting nothing
ok
Trying:
    [Integer(1),Integer(2),Integer(3)]###line 3:_sage_    >>> [1,2,3]
Expecting:
    [1, 2, 3]
ok
Trying:
    [Integer(1),###line 5:_sage_    >>> [1,
    Integer(2),Integer(3)
    ]
Expecting:
    [1, 2, 3]
ok
Trying:
    x=Integer(3); x###line 10:_sage_    >>> x=3; x
Expecting:
    3
ok
3 items had no tests:
    __main__
    __main__.change_warning_output
    __main__.warning_function
1 items passed all tests:
   5 tests in __main__.example_0
5 tests in 4 items.
5 passed and 0 failed.
Test passed.
	 [3.0 s]
 
----------------------------------------------------------------------
All tests passed!
Total time for all tests: 3.0 seconds
keshav@esterhazy /opt/sage/devel $ 

After I replace all "....: " with "... " using sed, everything seems to work fine.

It seems to me that the problem is that sage is not converting "....: " to "... ", which is what the standard python interactive prompt uses as a prefix for continued input lines. However, the sage prompt uses "....: ", so IMHO the doctesting framework should allow for this, so that lines from sage sessions can really be just dumped into a docstring without any further editing.

Change History (7)

Changed 12 years ago by Keshav Kini

Attachment: test.rst added

testing docstring

comment:1 Changed 12 years ago by Minh Van Nguyen

The Sage command line interface uses IPython, which is responsible for printing line continuation of the form "...:". It could be more appropriate to fix this issue upstream in the IPython project. We could send a patch upstream or find out how to get IPython to do line continuation of the form "...".

Changed 12 years ago by Keshav Kini

Attachment: continuation.patch added

patch to make sage recognize "....:" continuation line prefixes in doctests

comment:2 Changed 12 years ago by Keshav Kini

Authors: kini

Well, Sage already replaces "sage:" with ">>>", so why not replace "....:" with "..."? In any case, "....:" looks better, since it is the same length as "sage:", making the line prefix look more like a separate "column", as it should (note that it is "....:" with four periods, not "...:" as you have written).

Here is a patch which seems to work (at least the attached test.rst passes). What do you think? I'm currently running sage -testall with the patch applied to see if anything broke...

I'm new to sage_trac, so forgive me if setting my username as the "Author" was not the correct thing to do.

comment:3 Changed 12 years ago by Keshav Kini

OK, that didn't take all that long. All tests pass.

comment:4 Changed 12 years ago by Minh Van Nguyen

Description: modified (diff)

This is really a major change to how the doctesting framework behaves. The issue needs to be raised on sage-devel for some discussion.

comment:5 Changed 12 years ago by Leif Leonhardy

Authors: kiniKeshav Kini
Description: modified (diff)

Just fixed the https://.

And the Author(s) field. (This should contain the real name(s).)

Note: See TracTickets for help on using tickets.