HTML results for search_doc and search_src in Jupyter Notebook

[embray]

As pointed out in comment:4:ticket:24784, the search_doc and search_src functions, when run in the legacy Sage Notebook, produce HTML-formatted results, with working links to all the doc pages/source code containing search hits.

However, the Jupyter Notebook just displays the plain-text search results. As I pointed out in #24784, it is possible to return HTML-formatted results to Jupyter as well, in a mime-bundle. Jupyter will then determine which format to use depending on the context: plain text, or HTML, with a preference for HTML in the notebook.

This is easily fixed as demonstrated in the fix for #24784, but there are a few problems with the current HTML result formatting:

1. Most importantly, it produces URLs to search results that only work in the legacy notebook (which has its own URLs for hosting the Sage docs). Instead, I would propose that it produce links to doc.sagemath.org by default, but also have an optional offline=True option to produce file:/// URLs if the docs are available offline. The reason for making offline=False by default is that when running the Jupyter Notebook in a cloud-based setting, such as mybinder.org, producing file:/// links would not be useful. As mentioned below, we don't need this because we can serve the docs through /kernelspec/sagemath/doc. However, this should probably still check that the docs are installed locally so that they can be served by Jupyter. If not, it would still make sense to fall back on doc.sagemath.org.

2. The HTML that is currently output is very "web 1.0", using tags and features that are deprecated in HTML5. The output could easily be updated to modern HTML5 + CSS.

3. The output is also not very helpful. It simply displays links to the documentation pages in which results were found, but does not display the search results themselves. It would be easy, for each document in which there are results, to display the result snippets inline, perhaps with the search terms highlighted.

4. When linking to Sphinx docs it would also be good to include the highlight= query string parameter, in order to highlight the search term(s) in the page.

[nthiery]

For info: assuming the Sage documentation has been compiled, it's possible to access it through the relative URL /kernelspecs/sagemath/doc/. That's how the Help-> Reference Manual menu item and friends in the Jupyter notebook points to the documentation, and it works regardless of where the Jupyter server is running (locally, on binder, ...).

This should resolve the file:// dilemma. Well, again assuming that the doc is compiled; I don't know how strong an assumption this is. Somehow, the Help menu already makes that assumption, so presumably it's reasonable to make the same one elsewhere in Jupyter.

[jhpalmieri]

I think that in the past, people have also considered using Sphinx's search capabilities in search_doc, to get nicely formatted results with highlighted terms, etc. I don't think there was a ticket for this, but it might be something to consider.

[embray]

Oh, I was wondering if it were possible to serve kernel-specific URLs. That solves that.

[embray]

Replying to jhpalmieri:

I think that in the past, people have also considered using Sphinx's search capabilities in search_doc, to get nicely formatted results with highlighted terms, etc. I don't think there was a ticket for this, but it might be something to consider.

If nothing else I'm surprised we aren't using Sphinx's search indices.

[nthiery]

However, this should probably still check that the docs are installed locally so that they can be served by Jupyter. If not, it would still make sense to fall back on doc.sagemath.org.

Shall we go this way (it would indeed be nice to the user), it would be best to use the same logic for the Help menu. That is, have somewhere a "base_url_of_sage_doc()" that would take into account the presence of locally built doc and whether we are being served through a Jupyter kernel, or not; and use it in both places (and maybe elsewhere too).

[embray]

Replying to nthiery:

However, this should probably still check that the docs are installed locally so that they can be served by Jupyter. If not, it would still make sense to fall back on doc.sagemath.org.

Shall we go this way (it would indeed be nice to the user), it would be best to use the same logic for the Help menu. That is, have somewhere a "base_url_of_sage_doc()" that would take into account the presence of locally built doc and whether we are being served through a Jupyter kernel, or not; and use it in both places (and maybe elsewhere too).

I agree, it would be good to have a generic, reusable utility function to get the best available URL for the sage docs in any given context.

[embray]

Retargeting some of my tickets (somewhat optimistically for now).

[embray]

Removing most of the rest of my open tickets out of the 8.7 milestone, which should be closed.