Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#14370 closed defect (fixed)

Run IPython tests with --colors=NoColor

Reported by: jdemeyer Owned by: was
Priority: blocker Milestone: sage-5.9
Component: interfaces Keywords:
Cc: Merged in: sage-5.9.beta3
Authors: Jeroen Demeyer, Volker Braun Reviewers: John Palmieri, David Roe
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #12415, #14331, #13278 Stopgaps:

Description (last modified by vbraun)

On some systems:

Running doctests with ID 2013-03-27-15-38-22-527abcba.
Doctesting 2 files.
sage -t --long sage/misc/interpreter.py
**********************************************************************
File "sage/misc/interpreter.py", line 150, in sage.misc.interpreter.sage_prompt
Failed example:
    shell.run_cell('sage_prompt()')
Expected:
    u'sage'
Got:
    u'sage'
**********************************************************************
File "sage/misc/interpreter.py", line 566, in sage.misc.interpreter.interface_shell_embed
Failed example:
    shell.run_cell('List( [1..10], IsPrime )')
Expected:
    [ false, true, true, false, true, false, true, false, false, false ]
Got:
    [ false, true, true, false, true, false, true, false, false, false ]
**********************************************************************
2 items had failures:
   1 of   4 in sage.misc.interpreter.interface_shell_embed
   1 of   4 in sage.misc.interpreter.sage_prompt
    [107 tests, 2 failures, 3.3 s]
sage -t --long sage/misc/sage_extension.py
**********************************************************************
File "sage/misc/sage_extension.py", line 34, in sage.misc.sage_extension
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 39, in sage.misc.sage_extension
Failed example:
    shell.run_cell('%time 594.factor()')
Expected:
    CPU times: user ...
    Wall time: ...
    2 * 3^3 * 11
Got:
    CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s
    Wall time: 0.00 s
    2 * 3^3 * 11
**********************************************************************
File "sage/misc/sage_extension.py", line 75, in sage.misc.sage_extension.SageMagics.runfile
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 99, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 103, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    3
**********************************************************************
File "sage/misc/sage_extension.py", line 106, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    []
**********************************************************************
File "sage/misc/sage_extension.py", line 127, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    2
**********************************************************************
File "sage/misc/sage_extension.py", line 131, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    3
**********************************************************************
File "sage/misc/sage_extension.py", line 134, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    []
**********************************************************************
File "sage/misc/sage_extension.py", line 210, in sage.misc.sage_extension.SagePlainTextFormatter
Failed example:
    shell.run_cell('a = identity_matrix(ZZ, 2); [a,a]')
Expected:
    [
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
Got:
    [
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
**********************************************************************
5 items had failures:
   2 of  12 in sage.misc.sage_extension
   3 of  14 in sage.misc.sage_extension.SageMagics.attach
   3 of  14 in sage.misc.sage_extension.SageMagics.pre_run_code_hook
   1 of   9 in sage.misc.sage_extension.SageMagics.runfile
   1 of   5 in sage.misc.sage_extension.SagePlainTextFormatter
    [55 tests, 10 failures, 2.3 s]
----------------------------------------------------------------------
sage -t --long sage/misc/interpreter.py  # 2 doctests failed
sage -t --long sage/misc/sage_extension.py  # 10 doctests failed
----------------------------------------------------------------------
Total time for all tests: 5.7 seconds
    cpu time: 0.8 seconds
    cumulative wall time: 5.6 seconds

Also, on OS X 10.8:

sage -t --long devel/sage/sage/doctest/test.py
**********************************************************************
File "devel/sage/sage/doctest/test.py", line 17, in sage.doctest.test
Failed example:
    subprocess.call(["sage", "-t", "longtime.rst"], cwd=tests_dir)  # long time
Expected:
    Running doctests...
    Doctesting 1 file.
    sage -t longtime.rst
    [0 tests, ...s]
    ----------------------------------------------------------------------
    All tests passed!
    ----------------------------------------------------------------------
    ...
    0
Got:
    Running doctests with ID 2013-03-28-01-24-01-91891a00.
    Doctesting 1 file.
    [?1034hsage -t longtime.rst
        [0 tests, 0.0 s]
    ----------------------------------------------------------------------
    All tests passed!
    ----------------------------------------------------------------------
    Total time for all tests: 0.1 seconds
        cpu time: 0.0 seconds
        cumulative wall time: 0.0 seconds
    0
**********************************************************************

Apply 14370_ipython_nocolors_v2.patch, 14370_run_doctests_fix.patch

Attachments (3)

14370_ipython_nocolors.patch (7.8 KB) - added by jdemeyer 5 years ago.
14370_ipython_nocolors_v2.patch (6.8 KB) - added by jdemeyer 5 years ago.
14370_run_doctests_fix.patch (1.0 KB) - added by vbraun 5 years ago.
Initial patch

Download all attachments as: .zip

Change History (26)

comment:1 Changed 5 years ago by jdemeyer

  • Summary changed from Run IPython tests with --color=NoColors to Run IPython tests with --colors=NoColor

comment:2 Changed 5 years ago by jdemeyer

  • Status changed from new to needs_review

comment:3 Changed 5 years ago by jdemeyer

  • Description modified (diff)
  • Status changed from needs_review to needs_work

comment:4 Changed 5 years ago by vbraun

The OSX issue is the same as #12263, either set TERM=dumb or ellipsize the smm escape sequence that may or may not be there.

It would be nice to run with #14290 to verify that the problem actually is color and not some other escape sequence.

comment:5 Changed 5 years ago by jdemeyer

Excellent diagnosis Volker!

comment:6 Changed 5 years ago by jhpalmieri

On OS X 10.8: I ran make ptest, and all tests passed. Immediately afterward, I ran

./sage -tp devel/sage/sage/misc/interpreter.py devel/sage/sage/misc/sage_extension.py

and I got the failures listed here. Does that make any sense?

comment:7 Changed 5 years ago by vbraun

Maybe make ptest changes TERM somewhere on OSX? Try with #14290 to actually see the escape sequences, otherwise we can just guess.

comment:8 Changed 5 years ago by jhpalmieri

With #14290:

$ ./sage -tp devel/sage/sage/misc/interpreter.py devel/sage/sage/misc/sage_extension.py
Running doctests with ID 2013-03-28-10-54-43-7fc9b4da.
Doctesting 2 files using 2 threads.
sage -t devel/sage/sage/misc/sage_extension.py
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 34, in sage.misc.sage_extension
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 39, in sage.misc.sage_extension
Failed example:
    shell.run_cell('%time 594.factor()')
Expected:
    CPU times: user ...
    Wall time: ...
    2 * 3^3 * 11
Got:
    CPU times: user 0.00 s, sys: 0.00 s, total: 0.00 s
    Wall time: 0.00 s
    <CSI-0;31m><CSI-0m>2 * 3^3 * 11
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 75, in sage.misc.sage_extension.SageMagics.runfile
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 99, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 103, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    <CSI-0;31m><CSI-0m>3
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 106, in sage.misc.sage_extension.SageMagics.attach
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    <CSI-0;31m><CSI-0m>[]
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 127, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    2
Got:
    <CSI-0;31m><CSI-0m>2
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 131, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('a')
Expected:
    3
Got:
    <CSI-0;31m><CSI-0m>3
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 134, in sage.misc.sage_extension.SageMagics.pre_run_code_hook
Failed example:
    shell.run_cell('attached_files()')
Expected:
    []
Got:
    <CSI-0;31m><CSI-0m>[]
**********************************************************************
File "devel/sage/sage/misc/sage_extension.py", line 210, in sage.misc.sage_extension.SagePlainTextFormatter
Failed example:
    shell.run_cell('a = identity_matrix(ZZ, 2); [a,a]')
Expected:
    [
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
Got:
    <CSI-0;31m><CSI-0m>[
    [1 0]  [1 0]
    [0 1], [0 1]
    ]
**********************************************************************
5 items had failures:
   2 of  12 in sage.misc.sage_extension
   3 of  14 in sage.misc.sage_extension.SageMagics.attach
   3 of  14 in sage.misc.sage_extension.SageMagics.pre_run_code_hook
   1 of   9 in sage.misc.sage_extension.SageMagics.runfile
   1 of   5 in sage.misc.sage_extension.SagePlainTextFormatter
    [55 tests, 10 failures, 2.6 s]
sage -t devel/sage/sage/misc/interpreter.py
**********************************************************************
File "devel/sage/sage/misc/interpreter.py", line 150, in sage.misc.interpreter.sage_prompt
Failed example:
    shell.run_cell('sage_prompt()')
Expected:
    u'sage'
Got:
    <CSI-0;31m><CSI-0m>u'sage'
**********************************************************************
File "devel/sage/sage/misc/interpreter.py", line 566, in sage.misc.interpreter.interface_shell_embed
Failed example:
    shell.run_cell('List( [1..10], IsPrime )')
Expected:
    [ false, true, true, false, true, false, true, false, false, false ]
Got:
    <CSI-0;31m><CSI-0m>[ false, true, true, false, true, false, true, false, false, false ]
**********************************************************************
2 items had failures:
   1 of   4 in sage.misc.interpreter.interface_shell_embed
   1 of   4 in sage.misc.interpreter.sage_prompt
    [107 tests, 2 failures, 8.8 s]
----------------------------------------------------------------------
sage -t devel/sage/sage/misc/sage_extension.py  # 10 doctests failed
sage -t devel/sage/sage/misc/interpreter.py  # 2 doctests failed
----------------------------------------------------------------------
Total time for all tests: 8.9 seconds
    cpu time: 0.9 seconds
    cumulative wall time: 11.3 seconds

This command yields the same failures:

./sage -tp devel/sage/sage/misc/

This one passes all tests:

spkg/pipestatus "./sage -tp devel/sage/sage/misc 2>&1" "tee -a testmisc.log"

comment:9 Changed 5 years ago by jdemeyer

  • Status changed from needs_work to needs_review

comment:10 Changed 5 years ago by jhpalmieri

The patch doesn't apply cleanly to 5.9.beta1. What are the dependencies?

comment:11 Changed 5 years ago by jdemeyer

  • Dependencies set to #12415, #14331, #13278

comment:12 follow-up: Changed 5 years ago by jhpalmieri

  • Reviewers set to John Palmieri
  • Status changed from needs_review to positive_review

The code makes sense, and this seems to fix the problem. A minor fix:

  • sage/doctest/test.py

    diff --git a/sage/doctest/test.py b/sage/doctest/test.py
    a b  
    1313    sage: tests_dir = os.path.join(SAGE_SRC, 'sage', 'doctest', 'tests')
    1414    sage: tests_env = dict(os.environ)
    1515
    16 Unset :envvar:TERM when running doctests, see :trac:`14370`::
     16Unset :envvar:`TERM` when running doctests, see :trac:`14370`::
    1717
    1818    sage: try:
    1919    ....:     del tests_env['TERM']

Along these lines, is there a ticket to add files from the doctest directory to the reference manual?

Changed 5 years ago by jdemeyer

comment:13 in reply to: ↑ 12 Changed 5 years ago by jdemeyer

Replying to jhpalmieri:

Along these lines, is there a ticket to add files from the doctest directory to the reference manual?

See #14376, you can go ahead and make a patch for it.

Changed 5 years ago by jdemeyer

comment:14 Changed 5 years ago by jdemeyer

  • Status changed from positive_review to needs_work

comment:15 Changed 5 years ago by jdemeyer

  • Status changed from needs_work to needs_review

I changed the patch slightly, I think this is a better solution. Note the change to doctest/forker.py, needs review.

comment:16 Changed 5 years ago by jdemeyer

  • Description modified (diff)

comment:17 Changed 5 years ago by vbraun

I prefer v2, too. But run_doctests(sage.misc.interpreter) then still coughs up an escape code in shell.run_cell('List( [1..10], IsPrime )'), because that is not going through the doctest forker. I suppose run_doctests need not only disable color in the test shell but also the defaults if the doctest creates yet another shell.

Changed 5 years ago by vbraun

Initial patch

comment:18 Changed 5 years ago by vbraun

  • Description modified (diff)

comment:19 Changed 5 years ago by vbraun

I've added the patch that does that. Positive review to Jeroen's patch.

comment:20 Changed 5 years ago by roed

  • Status changed from needs_review to positive_review

Looks good to me.

comment:21 Changed 5 years ago by jdemeyer

  • Merged in set to sage-5.9.beta3
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:22 Changed 5 years ago by jdemeyer

  • Reviewers changed from John Palmieri to John Palmieri, David Roe

comment:23 Changed 5 years ago by jdemeyer

  • Authors changed from Jeroen Demeyer to Jeroen Demeyer, Volker Braun
Note: See TracTickets for help on using tickets.