Opened 23 months ago

# Improve introspection capabilities of Sage Jupyter Kernel

Reported by: Owned by: embray embray major sage-pending documentation egourgoulhon, zerline Erik Bray N/A u/embray/ticket-25015 (Commits) ec1e13a75695c0e1f1e0c9161cfc6b4cf3b4c687

Currently introspecting Sage objects in Jupyter (either the IPython console or the Notebook) with the obj? syntax is little different from the standard Python help(obj) output.

In particular on the Notebook this is not great because it means equations are not rendered--this is a major deficiency compared to SageNB.

The interface for Jupyter kernels does allow customizing what is sent back to the client upon object introspection. My understanding is that this can send back a multi-part MIME message in multiple formats (e.g. plain text and HTML) and the client will pick the most appropriate format to display. So on the Notebook, for example, it will prefer HTML-formatted help if any.

In this case the thing to do might be to look up the appropriate page for an object in Sage's reference docs, and return the HTML for that object, taking care to make sure that MathJax? rendering is applied, and any images returned as well. I'm not exactly sure about the details of making this work yet (images, for example, might have to be embedded as base64).

Relevant upstream issues:

### comment:1 Changed 23 months ago by embray

• Owner changed from (none) to embray

### comment:2 Changed 22 months ago by was

Related issue for CoCalc?: https://github.com/sagemathinc/cocalc/issues/2764

I'll implement something compatible with what you do here...

### comment:4 Changed 22 months ago by embray

I should add, that while I assigned myself this task OpenDreamKit is currently hiring for a position (we're actually interviewing candidates today) under which this work could very likely fall as well. So depending on how things go I might point someone else in the right direction to work on this...

### comment:5 Changed 21 months ago by embray

• Milestone changed from sage-8.2 to sage-8.3

### comment:6 Changed 19 months ago by embray

• Milestone changed from sage-8.3 to sage-8.4

### comment:8 Changed 15 months ago by embray

• Milestone changed from sage-8.4 to sage-8.5

### comment:9 Changed 14 months ago by embray

Really mostly what this has to do with is ensuring that help(obj) and obj? where obj is some object in the sage package displays the corresponding HTML documentation for that object if possible.

So we would need a way to find which page in the HTML docs contains the API documentation for that object, and return the HTML for that page (and specifically the section for that object) to Jupyter.

This would involve modifying what the Sage Jupyter kernel returns in response to inspect_request messages. Right now we just return a text/plain response containing the same text that would be displayed on the console. What we really want is to return a "MIME bundle": a dict of "text/plain" and "text/html".

The one thing I'm less sure about is the best way to format the HTML. Would it make sense to extract HTML from Sage's HTML docs (but in this case we also need to ensure that all the relevant stylesheets, javascript, etc. are loaded). Or do we do something like an <iframe> and just embed the URL to the appropriate doc page in that? The <iframe> approach might be simpler, but I don't know how well it will work. This might also be something worth asking on the Jupyter mailing list in case anyone has any advice or experience they could share...

### comment:10 Changed 14 months ago by jhpalmieri

The system with the legacy Sage notebook is to run sphinxify on the docstring to produce html documentation. Rather than try to extract from the built documentation, shouldn't you just keep doing this? The issue may be making sure that mathjax is loaded.

### comment:11 Changed 14 months ago by nthiery

I don't have an informed opinion for how to proceed. But either way, it's useful to recover the URL of the relevant chunk of Sage documentation, to at least include a "Jump to full/online documentation" link, enabling the user to browse the whole documentation from that entry point.

Thank you for pushing this forward!

Last edited 14 months ago by nthiery (previous) (diff)

### comment:12 Changed 13 months ago by embray

• Milestone changed from sage-8.5 to sage-8.7

Retargeting some of my tickets.

### comment:13 Changed 10 months ago by embray

• Milestone changed from sage-8.7 to sage-pending

Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.

### comment:14 Changed 6 months ago by embray

Finally have some progress on a working prototype of this (something I just felt compelled to work on today).

It turns out to be much more challenging that I originally expected :(

It turns out that the inspect_request message mentioned in the ticket description is only used, in the case of the IPython/Sage kernels, when pressing Shift-Tab to show a tooltip for the object under the cursor. It has nothing to do, in this case, with what happens when run obj?. That is handled entirely deep within the IPython input transformers, which converts this to a call to:

get_ipython().magic('pinfo obj')


In other words, obj? is equivalent to running the magic %pinfo obj. The implementation of the pinfo magic, then, is a complex affair which generates the help output as a string and passes that string to the IPython pager.

So the trick is the actually override how the pinfo magic works (which Sage already does a little bit) to have it pass the pager a "mime-bundle" containing both plain text and HTML, where the HTML has been generated by Sage's sphinxify() function.

On top of all that it requires a little more hacking to get the math displayed right. My prototype mostly works now, save for also loading the necessary CSS for the HTML docs to be displayed better.

Here's a screenshot showing an example of the in-progress work:

As you can see, one of the main problems is a lack of margins between paragraphs. Not shown in the screenshot, but code highlighting in the examples is broken. This is all due to not having the standard CSS load. Some non-standard LaTeX macros probably won't render either, though this is a more general problem with have right now in the Jupyter notebook.

Last edited 6 months ago by embray (previous) (diff)

### comment:15 Changed 6 months ago by embray

• Authors set to Erik Bray
• Branch set to u/embray/ticket-25015
• Commit set to 1a56e103c6dd8ed4c0d52b69d51a7a016d5702e9

Here's the in-progress version if anyone wants to try it out.

New commits:

 ​c09d2da Add strip_math_delims option to sphinxify so that its current default ​2800281 Add sphinxify_mimebundle function for sending docstrings as text+HTML ​1a56e10 Add the necessary default settings to enable HTML output in the Jupyter

### comment:16 Changed 6 months ago by embray

Another shortcoming I noticed is that sphinxify() makes no effort to resolve references, so if one docstring refers to another function or something, that reference won't be linked.

It would be nice if this could instead produce an actual link. Perhaps that should be tackled as a separate issue though.

### comment:17 follow-up: ↓ 19 Changed 5 months ago by egourgoulhon

Waouh! I've just checked it and it looks already very nice, even without the code highlighting in the doctests. Other shortcomings one might notice:

• the LaTeX macros \RR, \CC, etc. don't render
• the plots produced with sphinx_plot don't show up
• if one clicks on the button "Ouvrir le paginateur dans une fenêtre externe" ("Open the pager in external window" I guess --- sorry, my Jupyter is in French) in the top right of the help page, then the aspect change and the maths are printed twice, with standard LaTeX fonts and with some ugly fonts.

Anyway, all the above seem very minor inconveniences in perspective of the current state, which is ASCII-only documentation. So IMHO we could adopt your "prototype" as is and leave further improvements for other tickets.

### comment:18 Changed 5 months ago by embray

• Description modified (diff)

Thank you for testing it out!

I think fixing some of these issues is going to require new features in a Jupyter extension, in particular to load the JavaScript and CSS necessary fo fix some of those issues.

The double math-rendering thing is odd. It might be related also this upstream issue I opened. It hasn't gotten any notice yet though. Maybe it will get more if I go ahead and make a PR to propose a fix...

### comment:19 in reply to: ↑ 17 Changed 5 months ago by embray

• the LaTeX macros \RR, \CC, etc. don't render

By the way: This is a problem in the Jupyter notebook in general. Try making a text cell including them--they won't render properly there either, at least in my experience.

In SageNB, as well as in the Sphinx docs, we manually define these macros from:

def latex_extra_preamble():
r"""
Return the string containing the user-configured preamble,
sage_latex_macros, and any user-configured macros.  This is
used in the :meth:~Latex.eval method for the :class:Latex
class, and in :func:_latex_file_; it follows either
LATEX_HEADER or SLIDE_HEADER (defined at the top of this
file) which is a string containing the documentclass and standard
usepackage commands.

EXAMPLES::

sage: from sage.misc.latex import latex_extra_preamble
sage: print(latex_extra_preamble())
<BLANKLINE>
\newcommand{\ZZ}{\Bold{Z}}
\newcommand{\NN}{\Bold{N}}
\newcommand{\RR}{\Bold{R}}
\newcommand{\CC}{\Bold{C}}
\newcommand{\QQ}{\Bold{Q}}
\newcommand{\QQbar}{\overline{\QQ}}
\newcommand{\GF}[1]{\Bold{F}_{#1}}
\newcommand{\Zp}[1]{\Bold{Z}_{#1}}
\newcommand{\Qp}[1]{\Bold{Q}_{#1}}
\newcommand{\Zmod}[1]{\ZZ/#1\ZZ}
\newcommand{\CDF}{\Bold{C}}
\newcommand{\CIF}{\Bold{C}}
\newcommand{\CLF}{\Bold{C}}
\newcommand{\RDF}{\Bold{R}}
\newcommand{\RIF}{\Bold{I} \Bold{R}}
\newcommand{\RLF}{\Bold{R}}
\newcommand{\Bold}[1]{\mathbf{#1}}
<BLANKLINE>
"""
from sage.misc.latex_macros import sage_latex_macros
return "\n".join([_Latex_prefs._option['preamble'],
"\n".join(sage_latex_macros()),
_Latex_prefs._option['macros']])


Somehow we need to have the Sage Jupyter Notebook extension output some JavaScript? to define these in MathJax. I'm not really sure how that works though. Someone with more experience with both Juypter Extensions and MathJax should chime in on that...

### comment:20 Changed 5 months ago by was

Tiny side note:

This is a problem in the Jupyter notebook in general.

For what it is worth, these macros work in our implementation of Jupyter (for CoCalc?), with any kernel. Also, we use KaTeX by default, which is faster for math rendering. In addition, we cache rendering, so if you render the same formula again, it's much faster.

It might be easier just to get a PR into Jupyter that predefines some useful latex macros. You would also want to support nbconvert.

### comment:21 Changed 5 months ago by jhpalmieri

These macros are defined in src/sage/misc/latex_macros.py. That file has a function sage_mathjax_macros, which contains versions of these suitable for use with MathJax?, and that is what gets used in the legacy Sage notebook. I would hope there is a way to get the Jupyter notebook to use them.

### comment:22 Changed 4 months ago by zerline

A few elements about Sphinx code highlighting:

1) Sphinx uses pygments "friendly" style, with some very small changes.

2) At documentation build stage, Sphinx creates a CSS file called pygments.css:

from builders/html.py

    def copy_static_files(self):
# type: () -> None
try:
# copy static files
logger.info(bold(__('copying static files... ')), nonl=True)
ensuredir(path.join(self.outdir, '_static'))
# first, create pygments style file
with open(path.join(self.outdir, '_static', 'pygments.css'), 'w') as f:
f.write(self.highlighter.get_stylesheet())  # type: ignore


3) You can find the resulting CSS file in the HTML documentation tree, ie on

### comment:23 Changed 4 months ago by git

• Commit changed from 1a56e103c6dd8ed4c0d52b69d51a7a016d5702e9 to ec1e13a75695c0e1f1e0c9161cfc6b4cf3b4c687

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

 ​3067b7f Add strip_math_delims option to sphinxify so that its current default ​5ed3d00 Add sphinxify_mimebundle function for sending docstrings as text+HTML ​ec1e13a Add the necessary default settings to enable HTML output in the Jupyter

### comment:24 Changed 4 months ago by embray

I see; we probably don't want to rely on the full HTML docs being built/installed in order to load this stylesheet in the Jupyter kernel though; I'll look at that copy_static_files function and see if we can easily replicated its functionality, e.g., when installing the kernel.

New commits:

 ​3067b7f Add strip_math_delims option to sphinxify so that its current default ​5ed3d00 Add sphinxify_mimebundle function for sending docstrings as text+HTML ​ec1e13a Add the necessary default settings to enable HTML output in the Jupyter

### comment:25 Changed 3 months ago by egourgoulhon

May I ask about the status of this? It would be so nice to have it in the Sage 9.0 release. Even if it is not fully perfect, it is much much better than having the pure ASCII documentation in Jupyter notebooks...

### comment:26 Changed 6 weeks ago by embray

I haven't had time to work on it so if someone would like to take it over they are welcome to. I think improving some of the CSS stuff should still be done.

Since this might render some documentation less legible (e.g. due to the CSS or latex issues) it might be good to have a global function in Sage to enable/disable this (but it could be enabled by default in most cases I think...)

Note: See TracTickets for help on using tickets.