Sage: Ticket #27435: py3: failing doctest in graph_database.py with interactive_query
https://trac.sagemath.org/ticket/27435
<pre class="wiki">sage -t --long src/sage/graphs/graph_database.py
**********************************************************************
File "src/sage/graphs/graph_database.py", line 1081, in sage.graphs.graph_database.GraphDatabase.interactive_query
Failed example:
D.interactive_query(display_cols=['graph6', 'num_vertices', 'degree_sequence'], num_edges=5, max_degree=3)
Exception raised:
Traceback (most recent call last):
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/inspect.py", line 1119, in getfullargspec
sigcls=Signature)
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/inspect.py", line 2186, in _signature_from_callable
raise TypeError('{!r} is not a callable object'.format(obj))
TypeError: 0 is not a callable object
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/site-packages/sage/doctest/forker.py", line 671, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/site-packages/sage/doctest/forker.py", line 1095, in compile_and_execute
exec(compiled, globs)
File "<doctest sage.graphs.graph_database.GraphDatabase.interactive_query[1]>", line 1, in <module>
D.interactive_query(display_cols=['graph6', 'num_vertices', 'degree_sequence'], num_edges=Integer(5), max_degree=Integer(3))
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/site-packages/sage/graphs/graph_database.py", line 1087, in interactive_query
interact(f)
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/site-packages/sage/misc/decorators.py", line 343, in my_wrap
return func(*args)
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/site-packages/sagenb/notebook/interact.py", line 2636, in interact
(args, varargs, varkw, defaults) = inspect.getargspec(f)
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/inspect.py", line 1073, in getargspec
getfullargspec(func)
File "/Users/dcoudert/sage3/sage/local/lib/python3.6/inspect.py", line 1125, in getfullargspec
raise TypeError('unsupported callable') from ex
TypeError: unsupported callable
</pre>en-usSagehttps://trac.sagemath.org/chrome/site/logo_sagemath_trac.png
https://trac.sagemath.org/ticket/27435
Trac 1.1.6dcoudertThu, 07 Mar 2019 12:02:23 GMT
https://trac.sagemath.org/ticket/27435#comment:1
https://trac.sagemath.org/ticket/27435#comment:1
<p>
I don't know how to fix it.
</p>
Ticketgh-sheldoncooper07Fri, 08 Mar 2019 11:08:45 GMT
https://trac.sagemath.org/ticket/27435#comment:2
https://trac.sagemath.org/ticket/27435#comment:2
<p>
Hi, I am Piyush, I am new to sage, I was wondering if I could be of some help here.
</p>
<p>
I am not able to replicate this test. Is there something that I am missing? I am using Ubuntu 16.04
</p>
<pre class="wiki">$ ./sage -t --long src/sage/graphs/graph_database.py
Running doctests with ID 2019-03-08-11-03-04-40b4d48a.
Git branch: develop
Using --optional=dochtml,memlimit,mpir,python2,sage
Doctesting 1 file.
sage -t --long src/sage/graphs/graph_database.py
[50 tests, 0.49 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 0.6 seconds
cpu time: 0.5 seconds
cumulative wall time: 0.5 seconds
</pre>
TicketdcoudertFri, 08 Mar 2019 11:55:50 GMT
https://trac.sagemath.org/ticket/27435#comment:3
https://trac.sagemath.org/ticket/27435#comment:3
<p>
The problem is with Python 3. We are working hard on the transition from Python 2 to Python 3.
</p>
TicketembrayMon, 25 Mar 2019 10:56:15 GMTmilestone changed
https://trac.sagemath.org/ticket/27435#comment:4
https://trac.sagemath.org/ticket/27435#comment:4
<ul>
<li><strong>milestone</strong>
changed from <em>sage-8.7</em> to <em>sage-8.8</em>
</li>
</ul>
<p>
Ticket retargeted after milestone closed (if you don't believe this ticket is appropriate for the Sage 8.8 release please retarget manually)
</p>
Ticketgh-ed359Mon, 13 May 2019 04:42:35 GMT
https://trac.sagemath.org/ticket/27435#comment:5
https://trac.sagemath.org/ticket/27435#comment:5
<p>
I may have managed to fix this error. I don't know anything about the origins of the code but the error arises in in <code>graph_database._gen_interact_func</code> where it seems a string defining a function called <code>_</code> is run with <code>exec</code>.
</p>
<p>
I tried changing the name of the function that gets defined and ran into a different python3-related bug in the <code>sagenb</code> python package (maintained out of the main sage tree): it still uses <code>xrange</code> in <code>sagenb.notebook.interact</code> at line 1185.
</p>
<p>
I downloaded the source for that package, changed the uses of <code>xrange</code>, and now the doctest passes on my machine, but the bar for passing is simply emitting a string starting and finishing with <code><html></code> and <code></html></code>.
</p>
<p>
I'm not sure what to suggest for this ticket, the doctest in question has something to do with the old sage notebook (sagenb). The function <code>interactive_query</code> returns some html that doesn't appear do anything in jupyter, and I can't use the old notebook on python3 (an error related to Twisted). Is the old notebook formally deprecated? See <a class="closed ticket" href="https://trac.sagemath.org/ticket/22431" title="enhancement: upgrade sagenb and build sagenb in sage/python3 (closed: fixed)">#22431</a> and in particular the 'side remark' about the graph database explorer.
</p>
<p>
If my changes to <code>_gen_interact_func</code> and the <code>sagenb</code> package are wanted I can send patches, but as I can't run the notebook myself on python3 I can't even verify that it's a proper fix.
</p>
TicketdcoudertMon, 13 May 2019 06:14:36 GMTcc set
https://trac.sagemath.org/ticket/27435#comment:6
https://trac.sagemath.org/ticket/27435#comment:6
<ul>
<li><strong>cc</strong>
<em>dimpase</em> added
</li>
</ul>
<p>
Let's ask the author of <a class="closed ticket" href="https://trac.sagemath.org/ticket/22431" title="enhancement: upgrade sagenb and build sagenb in sage/python3 (closed: fixed)">#22431</a> if anything was said about <code>graph_editor</code> and interactive queries in <code>graph_database</code>.
</p>
<p>
Also I don't know if these tools have users. If not, we may simply deprecate the parts involving <code>sagenb</code>. Otherwise, someone with enough knowledge in notebooks should try to solve the issues.
</p>
TicketdimpaseMon, 13 May 2019 07:19:30 GMT
https://trac.sagemath.org/ticket/27435#comment:7
https://trac.sagemath.org/ticket/27435#comment:7
<p>
Replying to <a class="ticket" href="https://trac.sagemath.org/ticket/27435#comment:5" title="Comment 5">gh-ed359</a>:
</p>
<blockquote class="citation">
<p>
I may have managed to fix this error. I don't know anything about the origins of the code but the error arises in in <code>graph_database._gen_interact_func</code> where it seems a string defining a function called <code>_</code> is run with <code>exec</code>.
</p>
<p>
I tried changing the name of the function that gets defined and ran into a different python3-related bug in the <code>sagenb</code> python package (maintained out of the main sage tree): it still uses <code>xrange</code> in <code>sagenb.notebook.interact</code> at line 1185.
</p>
<p>
I downloaded the source for that package, changed the uses of <code>xrange</code>, and now the doctest passes on my machine, but the bar for passing is simply emitting a string starting and finishing with <code><html></code> and <code></html></code>.
</p>
<p>
I'm not sure what to suggest for this ticket, the doctest in question has something to do with the old sage notebook (sagenb). The function <code>interactive_query</code> returns some html that doesn't appear do anything in jupyter, and I can't use the old notebook on python3 (an error related to Twisted). Is the old notebook formally deprecated? See <a class="closed ticket" href="https://trac.sagemath.org/ticket/22431" title="enhancement: upgrade sagenb and build sagenb in sage/python3 (closed: fixed)">#22431</a> and in particular the 'side remark' about the graph database explorer.
</p>
<p>
If my changes to <code>_gen_interact_func</code> and the <code>sagenb</code> package are wanted I can send patches, but as I can't run the notebook myself on python3 I can't even verify that it's a proper fix.
</p>
</blockquote>
<p>
As long as these changes don't break sagenb on python2, it is fine.
Please send a PR on <a class="ext-link" href="https://github.com/sagemath/sagenb"><span class="icon"></span>https://github.com/sagemath/sagenb</a>
(perhaps they are already fixed in the 4 most recent open PRs there.)
</p>
<p>
As to functionality to be removed, I don't know.
</p>
Ticketgh-ed359Mon, 13 May 2019 18:06:20 GMTbranch set
https://trac.sagemath.org/ticket/27435#comment:8
https://trac.sagemath.org/ticket/27435#comment:8
<ul>
<li><strong>branch</strong>
set to <em>u/gh-ed359/py3__failing_doctest_in_graph_database_py_with_interactive_query</em>
</li>
</ul>
TicketkcrismanMon, 13 May 2019 18:41:49 GMTcc changed; commit set
https://trac.sagemath.org/ticket/27435#comment:9
https://trac.sagemath.org/ticket/27435#comment:9
<ul>
<li><strong>cc</strong>
<em>jdemeyer</em> added
</li>
<li><strong>commit</strong>
set to <em>2d05ce8822d9fd403ccd56dbffa90db9f1285367</em>
</li>
</ul>
<p>
Thanks. We may need to think about how to deal with this in the eventual removal of sagenb, as the interact should in principle still be able to work in e.g. Jupyter notebooks, but would have to import from elsewhere. Cc:ing Jeroen as he knows a lot about how that is implemented in Jupyter and where things might need to import from now.
</p>
<hr />
<p>
New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=2d05ce8822d9fd403ccd56dbffa90db9f1285367"><span class="icon"></span>2d05ce8</a></td><td><code>graph_database.py: fix doctest on python3</code>
</td></tr></table>
TicketdcoudertThu, 30 May 2019 15:59:52 GMT
https://trac.sagemath.org/ticket/27435#comment:10
https://trac.sagemath.org/ticket/27435#comment:10
<p>
Now the error is in sagenb (tested over 8.8.beta6).
</p>
<pre class="wiki">sage -t --long src/sage/graphs/graph_database.py
**********************************************************************
File "src/sage/graphs/graph_database.py", line 1082, in sage.graphs.graph_database.GraphDatabase.interactive_query
Failed example:
D.interactive_query(display_cols=['graph6', 'num_vertices', 'degree_sequence'], num_edges=5, max_degree=3)
Exception raised:
Traceback (most recent call last):
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1105, in compile_and_execute
exec(compiled, globs)
File "<doctest sage.graphs.graph_database.GraphDatabase.interactive_query[1]>", line 1, in <module>
D.interactive_query(display_cols=['graph6', 'num_vertices', 'degree_sequence'], num_edges=Integer(5), max_degree=Integer(3))
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/graph_database.py", line 1088, in interactive_query
interact(f)
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/misc/decorators.py", line 343, in my_wrap
return func(*args)
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sagenb/notebook/interact.py", line 2643, in interact
for i, arg in enumerate(args)]
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sagenb/notebook/interact.py", line 2643, in <listcomp>
for i, arg in enumerate(args)]
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sagenb/notebook/interact.py", line 3093, in render
to_value=self.__to_value, width=self.__width)
File "/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sagenb/notebook/interact.py", line 1185, in __init__
default_value = [[default_value[i * columns + j] for j in xrange(columns)] for i in xrange(rows)]
NameError: name 'xrange' is not defined
**********************************************************************
</pre>
TicketembrayFri, 14 Jun 2019 14:54:19 GMTmilestone deleted
https://trac.sagemath.org/ticket/27435#comment:11
https://trac.sagemath.org/ticket/27435#comment:11
<ul>
<li><strong>milestone</strong>
<em>sage-8.8</em> deleted
</li>
</ul>
<p>
As the Sage-8.8 release milestone is pending, we should delete the sage-8.8 milestone for tickets that are not actively being worked on or that still require significant work to move forward. If you feel that this ticket should be included in the next Sage release at the soonest please set its milestone to the next release milestone (sage-8.9).
</p>
TicketdcoudertSat, 15 Jun 2019 08:42:21 GMTmilestone set
https://trac.sagemath.org/ticket/27435#comment:12
https://trac.sagemath.org/ticket/27435#comment:12
<ul>
<li><strong>milestone</strong>
set to <em>sage-8.9</em>
</li>
</ul>
TicketjhpalmieriWed, 10 Jul 2019 18:52:16 GMT
https://trac.sagemath.org/ticket/27435#comment:13
https://trac.sagemath.org/ticket/27435#comment:13
<p>
If the problem is with sagenb, then can we mark the doctest as "# py2", since my understanding is that sagenb is not compatible with Python 3? That is, keep the proposed changes here, but also tag the doctest as py2 only.
</p>
TicketdimpaseWed, 10 Jul 2019 19:10:33 GMT
https://trac.sagemath.org/ticket/27435#comment:14
https://trac.sagemath.org/ticket/27435#comment:14
<p>
can't we just skip installing sagenb on Python3?
</p>
TicketjhpalmieriWed, 10 Jul 2019 19:24:55 GMT
https://trac.sagemath.org/ticket/27435#comment:15
https://trac.sagemath.org/ticket/27435#comment:15
<p>
I think that sagenb as a whole does not work with Python 3, but parts of it do, and parts are imported into the Sage library in various places. If we don't install it, some things may break. Some of the imports:
</p>
<pre class="wiki">src/sage/graphs/graph_editor.py: import sagenb.notebook.interact
src/sage/interacts/library.py: import sagenb.notebook.interact
src/sage/interacts/debugger.py: from sagenb.notebook.interact import slider, input_box, selector
</pre>
TicketdcoudertWed, 10 Jul 2019 20:36:46 GMT
https://trac.sagemath.org/ticket/27435#comment:16
https://trac.sagemath.org/ticket/27435#comment:16
<p>
I agree with the proposal of marking this doctest as <code># py2</code>, but we should also add that there is known issue with py3, no ? This way, we don't await for a decision about sagenb.
</p>
TicketdcoudertThu, 11 Jul 2019 09:44:46 GMTstatus, commit, branch changed; author set
https://trac.sagemath.org/ticket/27435#comment:17
https://trac.sagemath.org/ticket/27435#comment:17
<ul>
<li><strong>status</strong>
changed from <em>new</em> to <em>needs_review</em>
</li>
<li><strong>commit</strong>
changed from <em>2d05ce8822d9fd403ccd56dbffa90db9f1285367</em> to <em>10fd0e835c2a89f5e89b283c691d7a10abb99a70</em>
</li>
<li><strong>branch</strong>
changed from <em>u/gh-ed359/py3__failing_doctest_in_graph_database_py_with_interactive_query</em> to <em>public/graphs/27435_doctest_in_graph_database</em>
</li>
<li><strong>author</strong>
set to <em>Ewan Davies, David Coudert</em>
</li>
</ul>
<p>
I added proposed modification and pushed the code to a new branch in public. Feel free to modify it if needed.
</p>
<hr />
<p>
New commits:
</p>
<table class="wiki">
<tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=d5074f90df3c11fe1415a7235cf8ebf1c58ecdbf"><span class="icon"></span>d5074f9</a></td><td><code>trac #27435: Merged with 8.9.beta2</code>
</td></tr><tr><td><a class="ext-link" href="https://git.sagemath.org/sage.git/commit?id=10fd0e835c2a89f5e89b283c691d7a10abb99a70"><span class="icon"></span>10fd0e8</a></td><td><code>trac #27435: mark doctest as py2 only and add warning block</code>
</td></tr></table>
TicketjhpalmieriWed, 17 Jul 2019 21:02:22 GMT
https://trac.sagemath.org/ticket/27435#comment:18
https://trac.sagemath.org/ticket/27435#comment:18
<p>
This is fine with me. Anyone else have any comments? Objections to setting to positive review?
</p>
TicketchapotonThu, 18 Jul 2019 07:14:20 GMTstatus changed; reviewer set
https://trac.sagemath.org/ticket/27435#comment:19
https://trac.sagemath.org/ticket/27435#comment:19
<ul>
<li><strong>status</strong>
changed from <em>needs_review</em> to <em>positive_review</em>
</li>
<li><strong>reviewer</strong>
set to <em>Frédéric Chapoton, John Palmieri</em>
</li>
</ul>
<p>
ok..
</p>
TicketvbraunSat, 20 Jul 2019 09:10:54 GMTstatus, branch changed; resolution set
https://trac.sagemath.org/ticket/27435#comment:20
https://trac.sagemath.org/ticket/27435#comment:20
<ul>
<li><strong>status</strong>
changed from <em>positive_review</em> to <em>closed</em>
</li>
<li><strong>resolution</strong>
set to <em>fixed</em>
</li>
<li><strong>branch</strong>
changed from <em>public/graphs/27435_doctest_in_graph_database</em> to <em>10fd0e835c2a89f5e89b283c691d7a10abb99a70</em>
</li>
</ul>
Ticket