Opened 12 years ago

Closed 12 years ago

#5653 closed enhancement (fixed)

[with patch, positive review] display docstrings in the notebook using html and jsMath

Reported by: jhpalmieri Owned by: jhpalmieri
Priority: major Milestone: sage-4.1.1
Component: notebook Keywords:
Cc: mhansen Merged in: Sage 4.1.1.rc0
Authors: Tom Boothby, Evan Fosmark, John Palmieri, Mitesh Patel Reviewers: William Stein, Minh Van Nguyen, John Palmieri
Report Upstream: Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Status badges

Description

The attached patch shouldn't affect docstrings from the command line (well, except that it should replace 'a \times b' with 'a x b'). In the notebook, though:

identity_matrix(TAB

should pop open the usual docstring, but it's not in html <pre> format: see this picture. It's in a variable width font, with math typeset correctly -- see the $n \times n$ in the first line of the docstring -- and with example blocks in <pre> format.

This is marked "needs work" for two reasons, one of which is a mystery to me:

  1. (the mystery) start a fresh worksheet and type
    identity_matrix? [SHIFT-RETURN]
    

This will be typeset in <pre> format. Then type

identity_matrix? [TAB]

This is typeset nicely. I don't know what the difference is.

  1. Math is not handled properly. I have a hack in place to typeset inline math ("... `blah` ...") in the docstring, but not displayed math (directive ".. math::"). This is because I'm using docutils to convert the docstring to html, and docutils doesn't know about math. I would like to use Sphinx to do the conversion (in which case more math should be handled, and there would also be some syntax highlighting), but I don't know how to use Sphinx well enough to do that. I've posted a question to the sphinx-dev group, and if I hear anything, I'll post it here or update the ticket.

Please test out the patch, improve it, rewrite it, whatever.

(By the way, Tom Boothby deserves credit for getting jsMath to process the docstring.)

Attachments (13)

Fosmark_email.txt (5.0 KB) - added by mpatel 12 years ago.
Email exchange with Evan Fosmark
bsd.txt (1.5 KB) - added by mpatel 12 years ago.
Attachment to Evan Fosmark's email.
docstring.4.patch (54.4 KB) - added by jhpalmieri 12 years ago.
Apply only this patch, as it's cumulative. Rebased against 3.4.1.rc4
layout.html.pop (965 bytes) - added by mpatel 12 years ago.
Please see the comment
5653-cleanup.patch (25.1 KB) - added by boothby 12 years ago.
trac_5653_pretty_docstrings.patch (89.3 KB) - added by mpatel 12 years ago.
Conform to docbuild setup. This patch should be cumulative, against 3.4.2.
docstring.5.patch (25.0 KB) - added by jhpalmieri 12 years ago.
cumulative against 4.0.alpha0. Apply only this patch.
css.patch (1.3 KB) - added by whuss 12 years ago.
apply ontop of docstring.5.patch
css-new.patch (1.2 KB) - added by jhpalmieri 12 years ago.
apply on top of docstring.5.patch -- this replaces css.patch
docstring.6.patch (24.6 KB) - added by mpatel 12 years ago.
Rebase of docstring.5.patch + css-new.patch against v4.0.2 + #6307's 6307bis.patch. Apply only this patch.
trac_5653_pretty_docstrings_v7.patch (25.1 KB) - added by mpatel 12 years ago.
Re-base against v4.1 + #6307's "bis" and "reviewer" patches. Apply only this patch.
trac_5653-doc.patch (1.4 KB) - added by jhpalmieri 12 years ago.
apply on top of the other patch
trac_5653-manifest.patch (1.9 KB) - added by mvngu 12 years ago.
fix MANIFEST.in

Download all attachments as: .zip

Change History (79)

comment:1 Changed 12 years ago by jhpalmieri

Okay, here's a new version with a new picture. This uses Pat LeSmithe?'s version of set_introspect_html (see this thread), with some modifications.

Actually, this patch has three versions of set_introspect_html:

  • SPHINX_set_introspect_html (modification of Pat LeSmithe?'s version using Sphinx)
  • PYGMENTS_set_introspect_html (Pat LeSmithe?'s version using pygments -- doesn't do much for me)
  • DOCUTILS_set_introspect_html (the version from the first patch)

You can edit server/notebook/cell.py, changing the line

set_introspect_html = SPHINX_set_introspect_html

to test out each one.

comment:2 Changed 12 years ago by jhpalmieri

  • Summary changed from [with patch, needs work] display docstrings in the notebook using html and jsMath to [with patch, needs review] display docstrings in the notebook using html and jsMath

Here's a new patch incorporating Pat LeSmithe?'s latest ideas, plus a few new things. This handles everything pretty well, although it might be a bit slow since Sphinx creates various files. I think it's slower than the standard docstring display in the notebook, but on the other hand, I also think it's fast enough to be usable. If we ever figure out how to run Sphinx without reading and writing files, we can do that instead.

This latest version handles math well (via Sphinx), uses pygments to do syntax highlighting (via Sphinx), handles all of identity_matrix( TAB, identity_matrix? TAB, identity_matrix? SHIFT-RETURN, and identity_matrix??. There is now only one version of set_introspect_html: the Sphinx version.

One of Pat's recent posts said:

* If anyone's interested, here's a way to render an input cell as
syntax-highlighted HTML output:

from pygments import highlight
from pygments.lexers import PythonLexer
from pygments.formatters import HtmlFormatter
from pygments.styles import STYLE_MAP
from pygments.styles import get_style_by_name
from sphinx.highlighting import SphinxStyle

class colorize:
    def __init__(self, style=SphinxStyle):
        self.lexer = PythonLexer(encoding='chardet')
        self.formatter = HtmlFormatter(noclasses=True, style=style)
    def eval(self, s, globals, locals):
        return html(highlight(s, self.lexer, self.formatter))

Then put %colorize('colorful'), say, at the beginning of a cell and
evaluate it, e.g.,

%colorize('colorful')
def f(x):
  return x * x
f(3.0)

STYLE_MAP.keys() gives a list of styles.  This is adapted from
http://groups.google.com/group/sage-devel/msg/e53caae140cef7df . 

There is already a file and a function called "colorize", but I added code like this, so that you can do

%pygments('colorful')
def f(x):
  return x * x
f(3.0)

(I was thinking of a name like "html" or "jsmath": pygments is what processes the rest of the cell, so that's the name of the decorator.) With the patch, this is automatically available in the notebook.

I'm sure all of this could still use work, but I'll mark it as "needs review".

comment:3 follow-up: Changed 12 years ago by jhpalmieri

  • Owner changed from boothby to jhpalmieri
  • Status changed from new to assigned

A new patch to go on top of docstring-new.2.patch: this incorporates a contribution from Pat LeSmithe? which caches docstrings which have been processed by Sphinx and also cleans up the Sphinx processing (better conf.py and layout.html, so simpler set_instrospect_html). I added code to process docstrings which are not in ReST format (e.g. matrix_plot or preparse): these are displayed with headers like "File:" in boldface, just as for other docstrings, and with the body of the docstring in a <pre> block.

comment:4 in reply to: ↑ 3 Changed 12 years ago by mpatel

A new standalone patch, docstring.3.patch, tweaks the Sphinx processing code. The code now

  • Sets the cache directory once per session.
  • Deletes doctrees and environment pickles.
  • Tries to avoid crossed signals from [near] simultaneous calls to Sphinx.

It'd be great to learn how this performs under heavy server loads, especially when many people access the same (or different) docstrings at once.

The new patch also removes pygmentize.py. Eventually, it'll return in a separate ticket, along with nascent support for highlighting tracebacks and published input cells of various systems (e.g., python, latex, html, sh).

comment:5 Changed 12 years ago by jhpalmieri

Three comments: docstring.3.patch failed to apply cleanly against Sage 3.4.1.rc2 (someone else fixed a typo in support.py), so I've attached a rebased version. Also, you should post Mercurial patches, not just diff files, so there is an official record of who produced the patch.

The third comment is that it looks good to me. (I helped to write it, though, so I can't review it.)

comment:6 follow-up: Changed 12 years ago by mpatel

The patch's file locking algorithm is adapted from Evan Fosmark's Pythonic cross-platform file lock:

http://www.evanfosmark.com/2009/01/cross-platform-file-locking-support-in-python/

In an email, Evan has stated that this code falls under the BSD license. I think Evan should definitely get some credit.

comment:7 in reply to: ↑ 6 ; follow-up: Changed 12 years ago by mabshoff

Replying to mpatel:

In an email, Evan has stated that this code falls under the BSD license. I think Evan should definitely get some credit.

Could you post that email here for the record?

Cheers,

Michael

comment:8 Changed 12 years ago by mabshoff

By the way: I saw this in action today at UW's Sage status report and I was really impressed.

For credit: We have John, Mitel and Tom?

Cheers,

Michael

Changed 12 years ago by mpatel

Email exchange with Evan Fosmark

Changed 12 years ago by mpatel

Attachment to Evan Fosmark's email.

comment:9 in reply to: ↑ 7 ; follow-up: Changed 12 years ago by mpatel

Replying to mabshoff:

Could you post that email here for the record?

I've also repeated the license question in a comment on Evan Fosmark's original blog entry. I mentioned this in my email reply to Evan, so he may soon name the license explicitly on that page.

comment:10 in reply to: ↑ 9 Changed 12 years ago by mabshoff

Replying to mpatel:

I've also repeated the license question in a comment on Evan Fosmark's original blog entry. I mentioned this in my email reply to Evan, so he may soon name the license explicitly on that page.

Thanks. I do agree with you that Evan should be credited for his code, so we do need to add him to the list of people getting credit for this patch.

Cheers,

Michael

comment:11 follow-up: Changed 12 years ago by rbeezer

So, this is a Beautiful Thing. ;-) Very nicely formatted docstrings *and* in some cases, formatted source code as well. This is a great addition to the notebook for both users and developers.

I'm not qualified to do a review, but here are some comments based on a bit of testing.

  1. sudoku?? yields nicely formatted source code, but matrix?? does not. Maybe there's something about the source, not the patch?
  1. I'd documented sudoku.py a week ago, which is why I was experimenting with it.

(a) A URL in the references, unadorned, is a live link in the docstring view in the notebook. Very nice. Even nicer would be if a click on the link opened in a new tab/window.

(b) An error in the docstring (still) gets reported in the console where I invoked the notebook. So a person writing docstrings can debug this way.

  1. Would it be possible to build on this code to create a command to test doscstring formatting on a single file from the sage command line? E.g., sage -doc misc/sagedoc.py would build all the docstrings as HTML and view it in the developers's default web browser?
  1. Against 3.4.1.rc3 I had to fix a couple of rejects from the patch by hand.

Great work - I hope this sees a proper review soon.

Rob

comment:12 Changed 12 years ago by jhpalmieri

I don't know why matrix?? doesn't produce nice source code.

I'm attaching a rebased version of the patch, also.

The other questions are good ones, and I don't have answers for them, but I think they should be dealt with in a later ticket, once this is in.

Changed 12 years ago by jhpalmieri

Apply only this patch, as it's cumulative. Rebased against 3.4.1.rc4

comment:13 in reply to: ↑ 11 Changed 12 years ago by mpatel

Replying to rbeezer:

  1. sudoku?? yields nicely formatted source code, but matrix?? does not. Maybe there's something about the source, not the patch?

It seems so. Doing a search for just a space in emacs, or turning on whitespace-mode [1], reveals several stray TABs (I think) starting around the line

# Ensure we have a list of lists, each inner list

Replacing them with spaces or running whitespace-cleanup fixes the formatting. I apologize again for not providing a patch.

[1] http://www.emacswiki.org/emacs/WhiteSpace

comment:14 follow-up: Changed 12 years ago by jhpalmieri

This is a little silly, but untabify.patch removes all of the TABs I could find in the code. If it's too silly, ignore the patch; it should be independent of everything else (although according to mpatel, it should make some source code look better).

comment:15 in reply to: ↑ 14 ; follow-up: Changed 12 years ago by mabshoff

Replying to jhpalmieri:

This is a little silly, but untabify.patch removes all of the TABs I could find in the code. If it's too silly, ignore the patch; it should be independent of everything else (although according to mpatel, it should make some source code look better).

Yeah, we don't want any tabs in the Sage library, so I am happy to apply it. But I would suggest to move it to its own ticket since it can be reviewed independently of this patch. It might cause some merge problems with other patches, so I am not sure when the best time to apply it would be. I still have not pushed 3.4.1.final out the door and am thinking to get it in now since then 3.4.1 would not have any tabs :)

Also: This ticket is somewhat messy due to many attachments, so I am deleting some old version of the patch that have been replaced by the unfied patch.

Cheers,

Michael

comment:16 in reply to: ↑ 15 ; follow-up: Changed 12 years ago by jhpalmieri

Replying to mabshoff:

Yeah, we don't want any tabs in the Sage library, so I am happy to apply it. But I would suggest to move it to its own ticket since it can be reviewed independently of this patch. It might cause some merge problems with other patches, so I am not sure when the best time to apply it would be. I still have not pushed 3.4.1.final out the door and am thinking to get it in now since then 3.4.1 would not have any tabs :)

Okay, see #5848. I'll try to rebase it as needed...

Also: This ticket is somewhat messy due to many attachments, so I am deleting some old version of the patch that have been replaced by the unfied patch.

Great, thanks. Feel free to delete untabify.patch, too, since it's now in its own ticket.

John

comment:17 in reply to: ↑ 16 Changed 12 years ago by mabshoff

Replying to jhpalmieri:

Okay, see #5848. I'll try to rebase it as needed...

Thanks, I am tempted to merge that ticket right now into 3.4.1.final since I see no point in delaying this :).

Also: This ticket is somewhat messy due to many attachments, so I am deleting some old version of the patch that have been replaced by the unfied patch.

Great, thanks. Feel free to delete untabify.patch, too, since it's now in its own ticket.

Done.

John

Cheers,

Michael

comment:18 Changed 12 years ago by ddrake

Whoa, this is cool. I really like it!

I do have a tiny suggestion: can the basic text color be changed to black? I like reading black text better than blue.

Other than that, when doing identity_matrix? TAB and identity_matrix? SHIFT-RETURN both are typeset nicely, but with shift-return, I get a slightly thinner box with bigger text in the code blocks; with tab, the box is the full width of the worksheet. This is a little inconsistent, but both look very nice. (This is with the untabify patch applied.)

If I do matrix? TAB, it looks fine, but if I do matrix? SHIFT-RETURN, the box is wider, and is shifted a bit off to the right. I can read all the text, though.

Changed 12 years ago by mpatel

Please see the comment

comment:19 Changed 12 years ago by mpatel

I've attached a sample doc/introspect/templates/layout.html with "pop out" and "close" links. When testing it, be sure clear the doc cache, since the MD5 hash is applied only to the source.

Note: The JavaScript? code may well be atrocious.

comment:20 Changed 12 years ago by rbeezer

The layout.html.pop patch by mpatel works great for me, and the "close" link is almost as valuable as the "pop out" link. ;-) Limited testing prompts a few minor suggestions:

  1. I think of the action being described as "tear off" as being more suggestive than "pop out," but I'm not sure there is a real convention for naming this sort of thing.
  2. In Firefox I get an address bar for the new window, which points back to the whole worksheet. Can the Javascript create the window without an address bar?
  3. Would it be possible to have a link for "move to new tab"? I prefer to have all my browser "stuff" in one window with lots of tabs, rather than a proliferation of windows. Maybe others would feel similarly.

A great addition as-is, the above are just thoughts I had upon initial use. I also just sent a demonstration screenshot off to sage-devel.

comment:21 Changed 12 years ago by jhpalmieri

The pop-out thing is pretty neat, but the math is garbled in the popped-out window -- this is on Mac OS X, both Safari and Firefox. The spacing is wrong; it's as if the math takes no space, so the plain text is written on top of it. Check out identity_matrix? (in-line math) or A.iterates? (displayed math) for A a matrix. I can provide a screen shot if you can't reproduce it.

Would it be possible to have the new window have, as a title, the name of the object for which it is the docstring or source? That would be cool. Failing that, I would change "Foo!" to something else: "Bar!" maybe, or perhaps "Sage documentation"?

By the way, Rob, I see an address bar in Firefox but not in Safari. I don't know what implications this has, as far as being able to control it...

comment:22 Changed 12 years ago by rbeezer

The pop-out help is all legible for me, but now that you mention it, the math portions degrade as they move to the pop-out. This is Firefox on Ubuntu, reasonably up-to-date. For the displayed math in A.iterates() the math is no longer centered, the font is not the heavier-weight italicized font, and the superscripts are positioned correctly but are not in a smaller font. I notice in layout.html.pop the inclusion of the jsmath loading (jsmath/easy/load.js), but perhaps there is more information that needs to be available to this window for the math to typeset properly. Viewing the page source for the tear-off window seems to indicate the markup is coming through OK, but only main.css and jsmath/easy/load.js seem to be present for styling the text.

After viewing the page source for the notebook, on a hunch I tried replacing jsmath/easy/load.js by /jsmath/jsMath.js but there was no noticeable change. So perhaps the the pop-out needs more help, and the symptoms are just more severe on Mac OS X? Lets hope so.

I vote for "Bar!" ;-) It looks like "Sage Documentation" at minimum is an easy fix, but having something more specific would also be really useful.

comment:23 Changed 12 years ago by jhpalmieri

  • Cc mhansen added

comment:24 follow-up: Changed 12 years ago by mpatel

I've written a few JavaScript functions for "tear out" introspection, but I'm not sure where to put them. For now, I've put them in javascript_local/introspect.js and added

head += '<script type="text/javascript" src="/javascript_local/introspect.js"></script>\n'

to notebook.py, but the "local" repository seems to be gone. Alternatively, I can put them in javascript/introspect.js and use the "extcode" repository, but I think the latter will soon change significantly (see sage-devel).

js.py is another possibility, but I'd rather not put them there, since it might be useful to load introspect.js into a "torn out" window.

A related issue: It would be nice to have just one <script> element load jsMath with all of Sage's customizations (extensions, plug-ins, macros, etc.) into regular, published, printed, and docbrowser worksheets, as well as "torn out" introspection windows and offline documentation. Maybe easy/load.js is the right place (see #4714), but I think notebook.py generates the macro list on-the-fly.

comment:25 follow-up: Changed 12 years ago by jhpalmieri

I think the tear-out stuff should be part of a different ticket -- it enhances the other things here, but is a separate issue.

comment:26 in reply to: ↑ 25 ; follow-up: Changed 12 years ago by jhpalmieri

Replying to jhpalmieri:

I think the tear-out stuff should be part of a different ticket -- it enhances the other things here, but is a separate issue.

(Also, its presence here might slow down the refereeing process -- I think docstring.4.patch is ready to go and should get into Sage soon, while the tear-out stuff isn't there yet.)

comment:27 in reply to: ↑ 24 Changed 12 years ago by jason

Replying to mpatel:

A related issue: It would be nice to have just one <script> element load jsMath with all of Sage's customizations (extensions, plug-ins, macros, etc.) into regular, published, printed, and docbrowser worksheets, as well as "torn out" introspection windows and offline documentation. Maybe easy/load.js is the right place (see #4714), but I think notebook.py generates the macro list on-the-fly.

Davide (author of jsmath) sent these comments about the above paragraph:

Mpatel is right that jsMath/easy/load.js could be used for this. Rather than putting calls to jsMath.Setup.Script() or jsMath.Extention.Require() in-line in the document itself, these can be be put in the loadFiles array in easy/load.js. It is also possible to put the jsMath.Macro() calls into a file (say jsMath/local/sage.js), and add that file to the loadFiles array as well rather than put them in-line. Any sage-specific customization could go in local/sage.js as well. In the latest version of jsMath, there is even a macros array in easy/load.js for custom macros, so you would not even need an extra file for that. These features are, in fact, one of the important reasons for easy/load.js, so I hope you are able to take advantage of them.

comment:28 in reply to: ↑ 26 Changed 12 years ago by mpatel

Replying to jhpalmieri:

Replying to jhpalmieri:

I think the tear-out stuff should be part of a different ticket -- it enhances the other things here, but is a separate issue.

(Also, its presence here might slow down the refereeing process -- I think docstring.4.patch is ready to go and should get into Sage soon, while the tear-out stuff isn't there yet.)

Agreed. Please see #6001 for progress on the tear-out stuff.

comment:29 follow-up: Changed 12 years ago by jhpalmieri

  • Summary changed from [with patch, needs review] display docstrings in the notebook using html and jsMath to [with patch, needs work] display docstrings in the notebook using html and jsMath

A comment from mhansen: we shouldn't reproduce the files builder.py and conf.py. Instead, we should use the existing builder.py (in doc/common), and we should use doc/common/conf.py, putting modifications in introspect/conf.py (as with tutorial/conf.py, etc.). I couldn't figure out how to run Sphinx and tell it to do all of this, so mpatel, can you fix it?

Changed 12 years ago by boothby

Changed 12 years ago by mpatel

Conform to docbuild setup. This patch should be cumulative, against 3.4.2.

comment:30 in reply to: ↑ 29 Changed 12 years ago by mpatel

Replying to jhpalmieri:

A comment from mhansen: we shouldn't reproduce the files builder.py and conf.py. Instead, we should use the existing builder.py (in doc/common), and we should use doc/common/conf.py, putting modifications in introspect/conf.py (as with tutorial/conf.py, etc.). I couldn't figure out how to run Sphinx and tell it to do all of this, so mpatel, can you fix it?

I think so. Please see this patch, which should be inclusive. (I just exported changesets 12155-7 to the same file, in succession. Let me know if this is wrong.)

comment:31 Changed 12 years ago by mpatel

  • Summary changed from [with patch, needs work] display docstrings in the notebook using html and jsMath to [with patch, needs review] display docstrings in the notebook using html and jsMath

comment:32 Changed 12 years ago by jhpalmieri

(I just exported changesets 12155-7 to the same file, in succession. Let me know if this is wrong.)

It didn't work for me, but I prepared a new patch. This incorporates all of the earlier changes, plus just a little more: boothby's patch turned off the sphinx messages, and mine turns off all of the other messages (about the location of the introspection cache, for example). It provides a bad way to turn those messages back on, because I couldn't figure out a better way, despite trying for quite a while -- see the docstring for set_instrospect_html.

To produce the patch, here's what I did:

  1. imported the old patch "docstring.4.patch".
  1. used "hg rollback" to uncommit the changes
  1. performed the changes in your patch file (by hand, basically)
  1. "hg commit"

This produced an all-in-one patch.

Changed 12 years ago by jhpalmieri

cumulative against 4.0.alpha0. Apply only this patch.

comment:33 Changed 12 years ago by mpatel

Thanks very much for making a proper all-in-one patch, though I can't test it just now, for lack of time. How did tboothby combine two HG changesets into a single patch?

Perhaps we can use Python's logging module. I don't know how it affects performance, but with the right granularity and help from library authors, we may be able to provide "histories" for Sage computations, too.

I also found FirePython / FireLogger. Apparently, it can work with the logging module to send live messages to a panel in Firefox's Firebug extension. I don't know if the development version works with Firebug 1.4. I'll try to take a closer look soon. It might be useful to get feedback in the browser on "What is my Sage process doing now?"

comment:34 Changed 12 years ago by jhpalmieri

As far as the verbosity issue, I don't think it can be fixed cleanly now, so I propose that we leave it as is. My understanding is that when you run a worksheet, there are two Sage processes going on -- one for the worksheet itself, and one associated to the notebook server. The introspection stuff is handled by the notebook server, while any variables we might want to set are in the worksheet process; therefore we can't toggle verbosity from within a worksheet.

Changed 12 years ago by whuss

apply ontop of docstring.5.patch

comment:35 Changed 12 years ago by whuss

I think the documentation is currently much too colorful and therefore hard to read. I've attached a few tweaks to the css, which make the background lighter, change the text color to black, and also tone down the colors of the examples.

comment:36 Changed 12 years ago by jhpalmieri

I think the documentation is currently much too colorful and therefore hard to read. I've attached a few tweaks to the css, which make the background lighter, change the text color to black, and also tone down the colors of the examples.

I don't have strong color preferences, so I'm mostly okay with this, but I like having the EXAMPLES blocks set off with a different background color. So here's a slight variant on your css.patch; the only difference is that, while I'm keeping your light gray as the general background, the background for the examples is white.

Changed 12 years ago by jhpalmieri

apply on top of docstring.5.patch -- this replaces css.patch

comment:37 Changed 12 years ago by mpatel

The CSS overflow property can activate "inline" scrollbars:

div.docstring {
  overflow: auto;
}

comment:38 follow-up: Changed 12 years ago by jason

This is awesome!

When doing a double-question mark in Sage 4.0, I get:

Traceback (most recent call last):
  File "", line 1, in 
  File "/home/jason/.sage/sage_notebook/worksheets/admin/7/code/10.py", line 6, in 
    print _support_.source_code("sudoku", globals(), system="sage")
  File "/home/jason/sage/local/lib/python2.5/site-packages/sage/server/support.py", line 240, in source_code
    lines, lineno = sageinspect.sage_getsourcelines(obj, is_binary=False)
  File "/home/jason/sage/local/lib/python2.5/site-packages/sage/misc/sageinspect.py", line 568, in sage_getsourcelines
    return inspect.getsourcelines(obj)
  File "/home/jason/sage/local/lib/python/inspect.py", line 618, in getsourcelines
    lines, lnum = findsource(object)
  File "/home/jason/download/sage-4.0.rc2/local/lib/python2.5/site-packages/IPython/ultraTB.py", line 151, in findsource
IOError: could not get source code

However, this may not be an issue with this patch. The directories above are the directories in which I built sage. However, I moved Sage since then, so the directories above no longer exist. Why are the directory paths not updated when I move the Sage directory (and start up Sage)?

Changed 12 years ago by mpatel

Rebase of docstring.5.patch + css-new.patch against v4.0.2 + #6307's 6307bis.patch. Apply only this patch.

comment:39 in reply to: ↑ 38 Changed 12 years ago by mpatel

Replying to jason:

However, this may not be an issue with this patch. The directories above are the directories in which I built sage. However, I moved Sage since then, so the directories above no longer exist. Why are the directory paths not updated when I move the Sage directory (and start up Sage)?

I've experienced something similar after upgrading and moving the installation to a new home. I think this directory path issue is important but orthogonal to this ticket.

comment:40 Changed 12 years ago by jason

  • Summary changed from [with patch, needs review] display docstrings in the notebook using html and jsMath to [with patch, needs rebase] display docstrings in the notebook using html and jsMath

I hate to say this, but I get conflicts on my 4.1 tree. I applied the patches at #6307, then the patch here. Can you rebase this?

applying docstring.6.patch
patching file sage/misc/sagedoc.py
Hunk #1 FAILED at 19
Hunk #2 FAILED at 111
Hunk #4 FAILED at 143
3 out of 6 hunks FAILED -- saving rejects to file sage/misc/sagedoc.py.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
Errors during apply, please fix and refresh docstring.6.patch

It looks like this file was touched a lot in the last few weeks:

grout@tiny:~/sage/devel/sage/sage/misc$ hg log -d -30 sagedoc.py
changeset:   12667:e40ab81e2c6d
tag:         qtip
tag:         docstring.6.patch
tag:         tip
user:        Mitesh Patel <qed777@gmail.com>
date:        Sat Jun 20 09:07:16 2009 -0700
summary:     #5653, pretty docstrings

changeset:   12653:3534c8c4de50
user:        J. H. Palmieri <palmieri@math.washington.edu>
date:        Tue Jul 07 07:52:02 2009 -0700
summary:     fix for ref manual

changeset:   12616:0df4e2e58e79
user:        J. H. Palmieri <palmieri@math.washington.edu>
date:        Mon Jun 29 17:14:16 2009 -0700
summary:     add 1 to line numbers

changeset:   12615:50be675c81f1
user:        Dan Drake <drake@kaist.edu>
date:        Sun Jun 28 22:17:17 2009 -0700
summary:     [mq]: trac_6429_line_numbers.patch

changeset:   12614:6d776eac44c9
user:        J. H. Palmieri <palmieri@math.washington.edu>
date:        Sun Jun 28 21:29:54 2009 -0700
summary:     try to make search_src etc. less OS dependent

changeset:   12603:96d0f059d528
user:        J. H. Palmieri <palmieri@math.washington.edu>
date:        Thu Jun 25 17:13:23 2009 -0700
summary:     ref manual fixes

Changed 12 years ago by mpatel

Re-base against v4.1 + #6307's "bis" and "reviewer" patches. Apply only this patch.

comment:41 Changed 12 years ago by mpatel

I applied the rejected pieces by hand and exported anew. See trac_5653_pretty_docstrings_v7.patch. A diff of the diffs does not reveal any surprises, but let me know...

comment:42 follow-up: Changed 12 years ago by mpatel

Out of curiosity: Have any new TABs appeared in library .py and .pyx files?

comment:43 Changed 12 years ago by mpatel

  • Summary changed from [with patch, needs rebase] display docstrings in the notebook using html and jsMath to [with patch, needs review] display docstrings in the notebook using html and jsMath

comment:44 in reply to: ↑ 42 Changed 12 years ago by mpatel

Replying to mpatel:

Out of curiosity: Have any new TABs appeared in library .py and .pyx files?

Quite a few, according to

grep "`echo '\t'`" `find -name \*.pyx -o -name \*.py`

(from tcsh, on Linux). Or am I mistaken? If not, what is an efficient way to generate a patch like that at #5848?

I suppose we should make a [firm] request on sage-devel, too. Perhaps we can set up Mercurial to untabify or complain during commit or export events?

comment:45 Changed 12 years ago by jason

I definitely think you should bring this up on sage-devel.

comment:46 follow-up: Changed 12 years ago by jason

This looks fantastic.

This sometimes messes up docstrings that are not yet in rest. However, this also might spur action in getting the rest of the docs into rest format.

I'll try to look through this next week. I've also applied it to my tree so that I can test it normal usage.

comment:47 in reply to: ↑ 46 Changed 12 years ago by mpatel

Replying to jason:

This sometimes messes up docstrings that are not yet in rest. However, this also might spur action in getting the rest of the docs into rest format.

[Thanks!] Please let us know about outliers.

comment:48 follow-up: Changed 12 years ago by mpatel

Ideas for possible future improvements:

  • Cross-references. One approach: Post-process Sphinx's output in cell.py's set_introspect_html(). It may help to pass the module name as an argument.
  • Browse to the previous / next definition in the current module.
  • Live cells.
  • Tear-outs (cf. #6001).

Why don't we have figures in the reference manual?

comment:49 Changed 12 years ago by was

  • Summary changed from [with patch, needs review] display docstrings in the notebook using html and jsMath to [with patch, positive review] display docstrings in the notebook using html and jsMath

REFEREE REPORT:

Oh my god, this so totally rocks!!!

Regarding the actual code:

  • It's pretty robust. This is huge and worked no problem:
    sage.combinat.sloane_functions??
    

This just rocks.

comment:50 Changed 12 years ago by jason

Fantastic! I'm excited to see this in Sage!

comment:51 in reply to: ↑ 48 Changed 12 years ago by mpatel

Replying to mpatel:

Ideas for possible future improvements:

  • Cross-references. One approach: Post-process Sphinx's output in cell.py's set_introspect_html(). It may help to pass the module name as an argument.
  • Browse to the previous / next definition in the current module.
  • Live cells.
  • Tear-outs (cf. #6001).
  • Faster hashing of long docstrings. Perhaps we can first check the parent module's mtime, then hash if this test is inconclusive. Or use hg diff.
  • A tree-like browser.

comment:52 Changed 12 years ago by mpatel

  • Authors set to Tom Boothby, Evan Fosmark, John Palmieri, Mitesh Patel

comment:53 Changed 12 years ago by mvngu

  • Reviewers set to William Stein

comment:54 Changed 12 years ago by mvngu

  • Summary changed from [with patch, positive review] display docstrings in the notebook using html and jsMath to [with patch, needs work] display docstrings in the notebook using html and jsMath

With the patches at #6307, trac_5653_pretty_docstrings_v7.patch results in the following doctest failures:

sage -t -long devel/sage-main/sage/misc/sagedoc.py
**********************************************************************
File "/scratch/mvngu/release/sage-4.1.1.alpha0/devel/sage-main/sage/misc/sagedoc.py", line 156:
    sage: detex(r'More math: `x \mapsto y`.  {\bf Bold face}.')
Expected:
    'More math: `x  |-->  y`.  { Bold face}.'
Got:
    'More math: `x  |-->  y`.  {bf Bold face}.'
**********************************************************************
1 items had failures:
   1 of   7 in __main__.example_2
***Test Failed*** 1 failures.
For whitespace errors, see the file /scratch/mvngu/release/sage-4.1.1.alpha0/tmp/.doctest_sagedoc.py
	 [21.5 s]

<SNIP>

sage -t -long devel/sage-main/sage/misc/sageinspect.py
**********************************************************************
File "/scratch/mvngu/release/sage-4.1.1.alpha0/devel/sage-main/sage/misc/sageinspect.py", line 446:
    sage: sage_getdoc(identity_matrix)[5:43]
Expected:
    'Return the `n times n` identity matrix'
Got:
    'Return the `n x n` identity matrix ove'
**********************************************************************
1 items had failures:
   1 of   4 in __main__.example_8
***Test Failed*** 1 failures.
For whitespace errors, see the file /scratch/mvngu/release/sage-4.1.1.alpha0/tmp/.doctest_sageinspect.py
	 [2.2 s]

Changed 12 years ago by jhpalmieri

apply on top of the other patch

comment:55 Changed 12 years ago by jhpalmieri

I think "trac_5653-doc.patch" should fix the doctest failures.

comment:56 Changed 12 years ago by jhpalmieri

  • Summary changed from [with patch, needs work] display docstrings in the notebook using html and jsMath to [with patch, needs review] display docstrings in the notebook using html and jsMath

comment:57 Changed 12 years ago by jhpalmieri

(Since it got a positive review earlier, I think that only the doctest failures should be an issue now.)

comment:58 Changed 12 years ago by mvngu

  • Merged in set to Sage 4.1.1.alpha1
  • Resolution set to fixed
  • Reviewers changed from William Stein to William Stein, Minh Van Nguyen
  • Status changed from assigned to closed
  • Summary changed from [with patch, needs review] display docstrings in the notebook using html and jsMath to [with patch, positive review] display docstrings in the notebook using html and jsMath

With the patches

  1. trac_5653_pretty_docstrings_v7.patch and
  2. trac_5653-doc.patch

all doctests pass. Merged both patches.

comment:59 Changed 12 years ago by mpatel

Did trac_5653_pretty_docstrings_v7.patch merge without incident? The patch includes, e.g., doc/en/introspect/templates/layout.html, but it appears to be missing in v4.1.1.alpha1. Or am I mistaken?

comment:60 follow-up: Changed 12 years ago by mpatel

I regenerated the missing files with hg revert --all. I'm not sure what happened, since I have not touched them at all in v4.1.1.alpha1. I'll assume it was an isolated oddity. Or has MANIFEST.in struck again?

comment:61 in reply to: ↑ 60 Changed 12 years ago by mvngu

Replying to mpatel:

I regenerated the missing files with hg revert --all. I'm not sure what happened, since I have not touched them at all in v4.1.1.alpha1. I'll assume it was an isolated oddity. Or has MANIFEST.in struck again?

Yes. For this ticket and also for #4460. I'm about to restore these missing files by posting a patch against MANIFEST.in.

comment:62 follow-up: Changed 12 years ago by mvngu

For the patch trac_5653_pretty_docstrings_v7.patch, I assume that doc/en/introspect/static/empty is an empty file, right?

comment:63 in reply to: ↑ 62 Changed 12 years ago by mpatel

Replying to mvngu:

For the patch trac_5653_pretty_docstrings_v7.patch, I assume that doc/en/introspect/static/empty is an empty file, right?

I think it contains just a newline character. Thanks very much for making a patch. I should have known by now, especially given recent events. I suppose it's manifest density.

Changed 12 years ago by mvngu

fix MANIFEST.in

comment:64 Changed 12 years ago by mvngu

  • Merged in Sage 4.1.1.alpha1 deleted
  • Resolution fixed deleted
  • Status changed from closed to reopened
  • Summary changed from [with patch, positive review] display docstrings in the notebook using html and jsMath to [with patch, needs review] display docstrings in the notebook using html and jsMath

I'm reopening this ticket since the patch trac_5653_pretty_docstrings_v7.patch resulted in a corrupted repository in Sage 4.1.1.alpha1. This now should be resolved with the patch trac_5653-manifest.patch, which depends on the corresponding manifest patch for #4460. So apply patches in this order:

  1. the patches at #4460
  2. trac_5653_pretty_docstrings_v7.patch
  3. trac_5653-doc.patch
  4. trac_5653-manifest.patch

Of these, only trac_5653-manifest.patch needs to be reviewed.

comment:65 Changed 12 years ago by jhpalmieri

  • Summary changed from [with patch, needs review] display docstrings in the notebook using html and jsMath to [with patch, positive review] display docstrings in the notebook using html and jsMath

trac_5653-manifest.patch looks good to me.

comment:66 Changed 12 years ago by mvngu

  • Merged in set to Sage 4.1.1.rc0
  • Resolution set to fixed
  • Reviewers changed from William Stein, Minh Van Nguyen to William Stein, Minh Van Nguyen, John Palmieri
  • Status changed from reopened to closed

Merged:

  1. trac_5653_pretty_docstrings_v7.patch
  2. trac_5653-doc.patch
  3. trac_5653-manifest.patch
Note: See TracTickets for help on using tickets.