Ticket #7509 (closed enhancement: fixed)
notebook -- make it possible to debug Python code in the notebook, e.g., something like pdb that works in the notebook
| Reported by: | was | Owned by: | boothby |
|---|---|---|---|
| Priority: | major | Milestone: | sage-5.0 |
| Component: | interact | Keywords: | |
| Cc: | novoselt, whuss | Work issues: | |
| Report Upstream: | N/A | Reviewers: | David Roe |
| Authors: | William Stein, Wilfried Huss | Merged in: | sage-5.0.beta3 |
| Dependencies: | Stopgaps: |
Description (last modified by was) (diff)
NOTE: "sage -coverage" claims less than 100% coverage, but there is 100% coverage. It's a bug in the coverage script that it doesn't count doctests marked with >>>.
Attachments
-
trac_7509.patch
(10.3 KB) -
added by was 16 months ago.
- ready to go
-
trac_7509.2.patch
(10.5 KB) -
added by was 16 months ago.
- new version addressing the bug that was found ppurka -- apply only this
-
trac_7509.3.patch
(10.5 KB) -
added by was 16 months ago.
- address bug introduced by bugfix. apply only this.
-
trac_7509_syntax_highlight.patch
(4.5 KB) -
added by whuss 16 months ago.
- adds syntax highlighting
-
trac_7509.4.patch
(10.9 KB) -
added by was 16 months ago.
- apply only this -- includes shrinking the stack; also prints a message if source code not known.
-
trac_7509.5.patch
(11.2 KB) -
added by was 16 months ago.
- apply only this; addresses referee remarks.
Change History
comment:2 Changed 4 years ago by was
Paste the following code into any notebook cell (in any version of sage ever) and you have a debugger.
class debug:
"""
If you get a traceback in the notebook, type ``d=debug()`` right
after the error to create a debugger object on that traceback.
Using the debugger you can move and down the call stack, and
evaluate arbitrary code anywhere in the call stack (typically
to inspect the value of variables).
- printing ``d`` shows the call stack and some context
- ``d.up(n)`` (or ``d.u(n)``) moves up `n` frames in
the call stack
- ``d.down(n)`` (or ``d.d(n)``) moves down `n` frames
in the call stack
- ``d.list(n)`` (or ``d.l(n)``) displays `n` lines of source
code context around the current position in the stack trace
- ``d("some code")`` executes the given code in the
context of the current position in the stack trace
Notes:
- Input is not preparsed.
- You can define and work with many debug objects at the same time.
"""
def __init__(self):
import inspect, traceback
self.stack = inspect.getinnerframes(sys.last_traceback)
self.__curframe = len(self.stack) - 1
self.tb = traceback.format_tb(sys.last_traceback)
def curframe(self):
return self.stack[self.__curframe][0]
def frameno(self):
return self.__curframe
def __call__(self, line):
locals = self.curframe().f_locals
globals = self.curframe().f_globals
try:
code = compile(line + '\n', '<stdin>', 'single')
exec code in globals, locals
except:
t, v = sys.exc_info()[:2]
if type(t) == type(''):
exc_type_name = t
else: exc_type_name = t.__name__
print '***', exc_type_name + ':', v
def _highlight(self, line):
from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter
return highlight(line, PythonLexer(), HtmlFormatter())
def __repr__(self):
v = [' ' + str(i) + ': ' + ' '*(2*i) +
('\n'+' '*(2*i)).join(self.tb[i].splitlines())
for i in range(self.__curframe+1)]
if len(v) < len(self.stack):
v.append(' ............')
v.append(' ' + str(len(self.stack)-1) + ': (bottom frame)')
s = '<b>Traceback:</b>' + self._highlight('\n'.join(v)) + self._list()
return '<html>%s</html>'%s
def up(self, n=1):
self.__curframe -= n
if self.__curframe < 0:
self.__curframe = 0
u = up
def down(self, n=1):
self.__curframe += n
if self.__curframe >= len(self.stack):
self.__curframe = len(self.stack)-1
d = down
def _list(self, n=3):
curframe = self.curframe()
filename = curframe.f_code.co_filename
lineno = curframe.f_lineno
import linecache
code = ''.join('--> ' if i ==lineno else ' ' +
linecache.getline(filename, i, curframe.f_globals) for
i in range(lineno-n, lineno+n+1))
i = filename.rfind('site-packages/sage')
if i != -1:
fname = filename[i+len('site-packages/sage')+1:].rstrip('/')
file = '<a href="/src/%s" target="_new">%s</a>'%(fname,fname)
else:
file = '<pre>%s</pre>'%filename
t = """<b>Frame:</b> %s\n<b>File: </b>%s<hr>%s<hr>"""%(
self.frameno(),
file,self._highlight(code).strip())
return t
def list(self, n=3):
html(self._list(n))
l = list
comment:3 Changed 4 years ago by was
Here's a proof concept interact, assuming one did:
d1 = debug()
at some point.
@interact
def _(command=[None, 'up','down'], code=''):
if command == 'up':
d1.up()
if command == 'down':
d1.down()
if code:
print "<html>>>> %s"%code
d1(code)
print '<hr></html>'
print d1
comment:6 Changed 16 months ago by was
Steps to try this out interactively in the notebook:
- install the patch and rebuild Sage.
- Do anything that causes a traceback in the notebook, e.g., EllipticCurve([0,0])
- Type debug() in any other notebook cell. You can move up and down the call stack, adjust the amount of context, and evaluate code in the box. These are the *only* features currently supported -- no setting of breakpoints, stepping through execution, etc. But this is a very useful first step.
Source code display for Cython code isn't implemented yet. That's for another ticket. I think it's OK to put off, since the command line debugger doesn't support it anyways.
comment:8 follow-up: ↓ 9 Changed 16 months ago by ppurka
Should line 192 of your patch contain the string as r""" """?
t = """%s<hr>> %s"""%(code, file)
I see that "< >" constructs are eaten away by the browser. An example code which you can use to debug your Debug() (pun intended :)) is the following:
ReedSolomonCode(15,2,GF(16,'a')).minimum_distance()
You need to go to stack frame 8, and look at line 396 of the code that is printed.
Otherwise, it functions very nicely!
comment:9 in reply to: ↑ 8 Changed 16 months ago by was
Replying to ppurka:
Should line 192 of your patch contain the string as r""" """?
t = """%s<hr>> %s"""%(code, file)
No. r is *only* needed if your string contains backslashes and you don't want to escape them. It has nothing to do with HTML tags at all.
I see that "< >" constructs are eaten away by the browser.
Above t is an HTML string that I'm construting, and output within an <html> block for the notebook. It thus gets displayed properly.
An example code which you can use to debug your Debug() (pun intended :)) is the following:
ReedSolomonCode(15,2,GF(16,'a')).minimum_distance()You need to go to stack frame 8, and look at line 396 of the code that is printed.
Ah, now I see what you mean. The stupid source code display needs to have some HTML escaping done, e.g., < turned to <. There is a function in the notebook that does that properly, and I'll fix the patch accordingly. Thanks for finding a genuine bug. I'll also modify my example functions to include something like.
Otherwise, it functions very nicely!
Changed 16 months ago by was
-
attachment
trac_7509.2.patch
added
new version addressing the bug that was found ppurka -- apply only this
comment:10 follow-up: ↓ 11 Changed 16 months ago by was
ppurka -- please continue to review the patch now.
comment:11 in reply to: ↑ 10 Changed 16 months ago by ppurka
Replying to was:
ppurka -- please continue to review the patch now.
Yes. The listing is fixed now. But the hyperlink to the file is broken. I fail to see why this should happen, since the <hr> and filename are not escaped. (Tested this functionality on sagenb.org.)
Changed 16 months ago by was
-
attachment
trac_7509.3.patch
added
address bug introduced by bugfix. apply only this.
comment:12 Changed 16 months ago by was
Doh -- fixed now.
Changed 16 months ago by whuss
-
attachment
trac_7509_syntax_highlight.patch
added
adds syntax highlighting
comment:13 Changed 16 months ago by whuss
- Cc whuss added
This is very nice und useful.
The patch [trac_7509_syntax_highlight.patch] adds syntax highlighting to the stack frame, and removes the first few frames, since they don't show anything interesting.
The <hr>> things are also fixed. Apply after the first patch [trac_7509.patch].
comment:14 follow-up: ↓ 22 Changed 16 months ago by was
Hi whuss. Thanks. I'm reluctant to do syntax hilighting that way (the way you do in your patch, just on code), since it is very often completely wrong if we don't hilight the whole file. We need to hilight the whole file, then grab just the relevant part back. Interesting about removing the first few frames -- I was concerned about doing that; is this now consistent with %debug on the command line?
comment:15 Changed 16 months ago by roed
I think I agree that we should remove the first few frames. In the example I was playing with they were actually empty.
I don't know about syntax highlighting. I think it may be best to focus on in a later ticket.
Changed 16 months ago by was
-
attachment
trac_7509.4.patch
added
apply only this -- includes shrinking the stack; also prints a message if source code not known.
comment:16 Changed 16 months ago by was
I made a new patch:
- has the shrunk stack
- prints something when source code not available (yet)
- adds whuff as co-author.
comment:17 follow-up: ↓ 18 Changed 16 months ago by roed
- Status changed from needs_review to needs_work
- Reviewers set to David Roe
This is while debugging the example
EllipticCurve([0,0])
If you type in a command (e.g. "print ainvs"), execute it, switch to a different frame and press enter again, nothing happens: it doesn't execute the print statement in the new frame. You can get it to execute by hitting the button though. If you type in a different command, then it starts executing upon pressing enter again.
There are doctest failures in listing, since you updated the functions.
Another small request would be to add a doctest to the evaluate function that demonstrates the case that a user types in a command to the text box which raises an error.
comment:18 in reply to: ↑ 17 Changed 16 months ago by was
Replying to roed:
This is while debugging the example
EllipticCurve([0,0])If you type in a command (e.g. "print ainvs"), execute it, switch to a different frame and press enter again, nothing happens: it doesn't execute the print statement in the new frame. You can get it to execute by hitting the button though. If you type in a different command, then it starts executing upon pressing enter again.
This is a fundamental limitation of @interact right now. I can make a remark about this in the docstring.
I'll fix + improve the doctests and post a new patch.
There are doctest failures in listing, since you updated the functions.
Another small request would be to add a doctest to the evaluate function that demonstrates the case that a user types in a command to the text box which raises an error.
Changed 16 months ago by was
-
attachment
trac_7509.5.patch
added
apply only this; addresses referee remarks.
comment:20 Changed 16 months ago by roed
Testing...
comment:21 Changed 16 months ago by roed
- Status changed from needs_review to positive_review
All tests pass.
comment:22 in reply to: ↑ 14 Changed 16 months ago by whuss
Replying to was:
Hi whuss. Thanks. I'm reluctant to do syntax hilighting that way (the way you do in your patch, just on code), since it is very often completely wrong if we don't hilight the whole file. We need to hilight the whole file, then grab just the relevant part back.
I want to avoid highlighting the whole file, since this can be quite slow. On my computer it takes about 2 seconds to load and highlight graphs/generic_graph.py.
I think the main problem when one gets wrong highlighting is when the code fragment starts in the middle of a docstring. At ticket:12451 there is an improved patch which detects when the code fragments starts in the middle of a tripple quoted string, and does the correct highlighting in this case.
comment:23 Changed 16 months ago by jdemeyer
- Status changed from positive_review to closed
- Resolution set to invalid
- Milestone changed from sage-5.0 to sage-duplicate/invalid/wontfix
Please report this patch to the SageNB developers
comment:24 Changed 16 months ago by jdemeyer
comment:25 Changed 16 months ago by jdemeyer
- Status changed from closed to new
- Resolution invalid deleted
- Component changed from notebook to interact
- Milestone changed from sage-duplicate/invalid/wontfix to sage-5.0
Apologies, this is actually independent of the notebook.
comment:27 Changed 16 months ago by jdemeyer
- Status changed from needs_review to positive_review
- Authors set to William Stein, Wilfried Huss
comment:28 Changed 16 months ago by jdemeyer
- Status changed from positive_review to closed
- Resolution set to fixed
- Merged in set to sage-5.0.beta3
comment:29 follow-up: ↓ 30 Changed 16 months ago by aranc
i think there is a minor typo in the docstring line 264
missing word "up": Using the debugger you can move <up> and down the stack frame and
--
i'm not sure what is the preferred convention for fixing typos, i can make a patch for this isolated line.
comment:30 in reply to: ↑ 29 Changed 15 months ago by was
- Description modified (diff)
