Opened 19 months ago
Last modified 4 weeks ago
#25837 needs_work task
Make the sage notebook optional
Reported by:  ghtimokau  Owned by:  

Priority:  major  Milestone:  sage9.1 
Component:  packages: standard  Keywords:  notebook, sagenb 
Cc:  embray, vbraun, jdemeyer, kcrisman, fbissey, arojas, ghtimokau, infinity0, novoselt, pcpa, nthiery, slelievre, saraedum, schilly, strogdon, thansen, tmonteil, was  Merged in:  
Authors:  Reviewers:  Dima Pasechnik  
Report Upstream:  N/A  Work issues:  
Branch:  u/chapoton/25837 (Commits)  Commit:  ec7931a58d5ffec3b08ea8274911c4e17e6a80f1 
Dependencies:  Stopgaps: 
Description (last modified by )
If you're coming from the deprecation notice in the notebook
The legacy sage notebook has been effectively deprecated for quite some time. We are now making that official. But that does not mean that sage will not have notebook functionality. In fact, most of the functionality of the sage notebook can already be replaced by using the Juypter Notebook instead. For an introduction on how to use the new notebook and convert existing notebooks, see the documentation.
There are *some* features that are not yet ported to the new notebook hovever. We will not completely stop supporting the legacy notebook until those issues are resolved. Those features are listed below. If there is some feature you're missing that is not listed or you want to give general feedback, feel free to leave a comment on this ticket. If you do not have an account yet, you can log in with a GitHub account.
Implementation details
Reasons to make sagenb optional
 most of its functionality has been replaced for a while by Jupyter
 it is not actively maintained  merely on life support
 it has various open issues (looking at the "Critical" label in the github repo)
 unmaintained dependencies (https://github.com/sagemath/sagenb/issues/440#issuecomment381583952)
 incompatibility with the latest flaskbabel (https://github.com/sagemath/sagenb/issues/437)
 incompatibility with python3 (https://github.com/sagemath/sagenb/issues/343)
 an XSS vulnerability, unfixed for 3.5 years (https://github.com/sagemath/sagenb/issues/319)
 and for me personally: I don't want to package it. I feel like it causes an unproportional amount of issues considering its relatively low value in todays sage.
Reasons to keep sagenb around
 there are still some features depending on it
 there might still be people using it. An example would be @fbissey's university: https://trac.sagemath.org/ticket/25382#comment:16
In making it optional, we still make it possible to use it for now. An when it inevitably breaks further, the people relying on it can weigh the effort of keeping it alive themselves against the effort of switching to the new notebook.
Tickets and issues tracking progress/going in that direction
Necessary to keep all tested behaviour in tact without sagenb ("pass the doctests")
 port/remove the graph editor
 port/remove the graph database navigator
 port/remove the interactive debugger
 #25382: do not include sagenb docs in sage
Misc changes
 #25852: add a deprecation warning to the sage notebook
 actually make sagenb optional
Functionality sagenb currently does better than Jupyter
 #25015: object introspection (
<obj>?
) in the Jupyter notebook does not display HTMLformatted docs (esp. with latex formatting), while sagenb does
 #24784:
search_doc
does not work in the Jupyter notebook
 #9048:
automatic_names
only works in sagenb
 Jupyter's converting to PDF "kind of sucks" (see #23330)
 #26101: With Jupyter,
view(...)
currently displays the pdf in an external pdf viewer, launched from the server. An inline view as in sagenb is nicer and would work remotely as well.
 Jupyter does not scroll large output (e.g. matrices) well; see this post on Stackoverflow
 probably supports matrix input, which does work in Sage cell server but not in Jupyter as of this Sage support discussion
Otherwise relevant
 #22431: first step in making sagenb python3 compatible
 https://wiki.sagemath.org/IpythonNotebook (which features of the Sage notebook are missing in the jupyter notebook, to be updated)
 MathJax? fontsI don't know if it was possible to change the font in the legacy notebook either, but it's not easy to configure a different font in Jupyter notebook. See e.g.:
Change History (51)
comment:1 followup: ↓ 3 Changed 19 months ago by
comment:2 Changed 19 months ago by
I would also say that something "official" when one uses sagenb even when directly that says it is deprecated is necessary.
comment:3 in reply to: ↑ 1 Changed 19 months ago by
 Description modified (diff)
comment:4 Changed 19 months ago by
I modified the ticket description to incorporate (read: shamelessly copy) your additions.
It would be nice if we could get the deprecation warning into 8.3 so that users will have at least one release to prepare. Thats a very optimistic estimate of course, but getting it in 8.3 would be nice either way.
comment:5 Changed 19 months ago by
 Description modified (diff)
I have opened #25852 for that. I have only ever actually started the notebook once. Do you have an idea where best to add such a message? Just print it on the command line?
comment:6 Changed 19 months ago by
Probably on the notebook itself as well, which would require changes in sagenb.
comment:7 Changed 19 months ago by
 Cc arojas ghtimokau infinity0 novoselt pcpa nthiery slelievre saraedum schilly strogdon thansen tmonteil was added
 Keywords sagenb added
cc a few people who package Sage in various ways
comment:8 Changed 19 months ago by
Another issue that was brought up here: https://ask.sagemath.org/question/43051/filedownloadaslatex/
Converting a Jupyter notebook to PDF kind of sucks at the moment for notebooks that contain a lot of math in latex. There are two main cases in the notebook where latex can appear, and both have issues in sage:
1) If an object output from a code cell has _render_latex_
then the notebook will display the rendered latex with mathjax. However, when converting the notebook to PDF, nbconvert just takes the plaintext representation of the output, not the latex, and puts it in a verbatim block. Not good, especially not for Sage.
2) If a markdown cell contains equations in latex, nbconvert *will* render them correctly, but only if you have pandoc installed. This is already a known issue, but since pandoc does not come with Sage and is not a dependency this makes for a sucky user experience for something that should Just Work.
I'm not sure what to do about the first case; it seems like an issue for nbconvert but I haven't explored it too deeply. For the latter case (which probably also affects the first case) I wonder if we shouldn't just make pandoc another sage dependency.
comment:9 Changed 18 months ago by
 Description modified (diff)
I've updated the ticket. The flask issue was resolved (thanks @fchapoton!), I added the pdf complaint and improved formatting.
comment:10 Changed 18 months ago by
 Description modified (diff)
comment:11 Changed 18 months ago by
 Description modified (diff)
I've added a section intended for end users. Please improve on that if you have more knowledge about sagenb or the Jupyter Notebook, which you probably do.
comment:12 Changed 18 months ago by
 Description modified (diff)
comment:13 Changed 18 months ago by
 Description modified (diff)
comment:14 Changed 18 months ago by
 Description modified (diff)
comment:15 followup: ↓ 16 Changed 18 months ago by
Those features are listed below.
Where is the list? I can't find it.
I touched on this on page 26 of a talk I gave yesterday (https://share.cocalc.com/share/6cd832d3c52341e39e54c8f2d2e8fa2a/travel/2018/201807icms/talks/cocalc/slides/slides.html#1)
One example is that if you start running:
`
for i in range(10):
print(i) sys.stdout.flush() time.sleep(1)
`
then close and open your browser (to simulate a network failure, say), you
should not loose output. In Jupyter, you do.
comment:16 in reply to: ↑ 15 Changed 18 months ago by
Replying to was:
Those features are listed below.
Where is the list? I can't find it.
It's in the description of the ticket, but perhaps that still needs more reformatting, because there are a number of thing "below" that sentence that include, but are not limited to, missing features.
Most of it falls under the heading "Functionality sagenb currently does better than Jupyter" though that list is far from complete. There's also a link to a wiki page listing other issues.
comment:17 Changed 17 months ago by
 Description modified (diff)
comment:18 Changed 17 months ago by
 Description modified (diff)
comment:19 Changed 17 months ago by
"#24784: search_doc does not work in the Jupyter notebook" is now ready for review.
comment:20 followup: ↓ 21 Changed 17 months ago by
Could someone (having the required rights) please now merge the 4 top pull requests in sagenb github, including the one related to the present ticket ?
comment:21 in reply to: ↑ 20 Changed 15 months ago by
comment:22 Changed 14 months ago by
 Branch set to u/chapoton/25837
 Commit set to ec7931a58d5ffec3b08ea8274911c4e17e6a80f1
New commits:
ec7931a  make sagenb optional

comment:23 Changed 11 months ago by
 Description modified (diff)
comment:24 Changed 11 months ago by
 Milestone changed from sage8.3 to sage8.8
comment:25 Changed 11 months ago by
remaining imports in 8.7.b6 :
git grep "import.*sagenb" src/ src/doc/en/developer/sagenb/index.rst:The most important section involves how to update your new sagenb src/sage/all.py:lazy_import('sagenb.notebook.notebook_object', 'notebook') src/sage/all.py:lazy_import('sagenb.notebook.notebook_object', 'inotebook') src/sage/all.py:lazy_import('sagenb.notebook.sage_email', 'email') src/sage/all.py:lazy_import('sagenb.notebook.interact', 'interact') src/sage/graphs/graph_editor.py: import sagenb.notebook.interact src/sage/interacts/library.py: import sagenb.notebook.interact
comment:26 followup: ↓ 29 Changed 10 months ago by
Apparently this question implies that we should add "can't do the table command" in Jupyter. Is that accurate?
comment:27 Changed 10 months ago by
It works for me.
comment:28 Changed 10 months ago by
 Status changed from new to needs_review
comment:29 in reply to: ↑ 26 Changed 10 months ago by
Replying to kcrisman:
Apparently this question implies that we should add "can't do the table command" in Jupyter. Is that accurate?
Now tracked at #27656.
comment:30 followup: ↓ 31 Changed 9 months ago by
 Reviewers set to Dima Pasechnik
 Status changed from needs_review to positive_review
Does anything stop this from going forward?
comment:31 in reply to: ↑ 30 Changed 9 months ago by
Does anything stop this from going forward?
Well, does Jupyter actually do all this stuff yet or not?
Also, for people who download a new Sage (which many do, as opposed to pulling and upgrading), it's not clear to me what happens when they try to use the notebook export functionality (which should still be the default as of now). How do they acquire this optional package if they have a binaryonly Sage? Or would the export still accurately export to Jupyter anyway?
comment:32 followup: ↓ 35 Changed 9 months ago by
 Status changed from positive_review to needs_info
IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.
Is there a reason to rush this forward? If so, what tickets could use my help?
comment:33 Changed 9 months ago by
 Description modified (diff)
comment:34 Changed 7 months ago by
 Milestone changed from sage8.8 to sage8.9
Tickets still needing working or clarification should be moved to the next release milestone at the soonest (please feel free to revert if you think the ticket is close to being resolved).
comment:35 in reply to: ↑ 32 ; followup: ↓ 37 Changed 4 months ago by
Replying to embray:
IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.
Is there a reason to rush this forward? If so, what tickets could use my help?
If SageNB doesn't work with Python 3, then we will need to drop it when we make the switch. Since the switch is coming up, we should officially deprecate this as soon as possible. If we manage to switch to Python 3 by January 1, we already won't make the required (by Sage policy) 6 months of deprecation time.
comment:36 Changed 4 months ago by
 Status changed from needs_info to needs_work
SageNB won't be ported to Python 3, it's been clear for ages I think.
I think this patch is potentially breaking the current notebooklaunching setup,
as we don't want to have a not always installed package pointed to by
Run the old Sage Notebook
menu item.
The latter should not be shown if the package is not installed. In this case instead a message saying something like "Install sagenb optional package for the old SageNB" should be there.
comment:37 in reply to: ↑ 35 Changed 4 months ago by
Replying to jhpalmieri:
Replying to embray:
IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.
Is there a reason to rush this forward? If so, what tickets could use my help?
If SageNB doesn't work with Python 3, then we will need to drop it when we make the switch. Since the switch is coming up, we should officially deprecate this as soon as possible. If we manage to switch to Python 3 by January 1, we already won't make the required (by Sage policy) 6 months of deprecation time.
I added a deprecation warning close to a year ago: https://trac.sagemath.org/ticket/25852
So I think we're good on that front if we decide to drop it with the first python3 release. Feature parity is a different issue though.
comment:38 Changed 4 months ago by
I have made a pull request to change the front page of the exporter.
comment:39 Changed 4 months ago by
This pull request have been merged.
comment:40 Changed 3 months ago by
We will not completely stop supporting the legacy notebook until those issues are resolved.
Did those issues get resolved, and/or will they soon? automatic_names
for instance would be an actual regression. Does Jupyter now support matrix input for interacts? Just wondering, before the switch is actually pulled ...
comment:41 Changed 3 months ago by
And was the graph stuff ever removed as is suggested in the description?
comment:42 Changed 3 months ago by
When we talk about regressions, I think the context is: we are imagining people converting their notebooks from SageNB to Jupyter and not using SageNB any more (as opposed to: using SageNB for some tasks, Jupyter for others). In that context, it is also worthwhile to ask what features Jupyter has that SageNB doesn't. I don't think we are just losing things by moving from SageNB to Jupyter; aside from Python 3 compatibility, I think we gain some things by moving to Jupyter. So it is a tradeoff. At what point does the balance tip to the Jupyter side? Maybe the Python 3 support is enough on its own, in fact.
comment:43 Changed 3 months ago by
Maybe the Python 3 support is enough on its own, in fact.
Obviously that is the consensus decision. I'm just quoting this ticket right now when I mention these things, and would hate to have them disappear for all time. But as you said elsewhere, someone has to work on them.
comment:44 Changed 2 months ago by
A functionality that Jupyter is currently missing over sagenb is the rich text editor. This may possibly be resolved by:
comment:45 Changed 2 months ago by
the most pressing issue is inability to normally use jsmol in Jupyter notebooks, needed to export 3d drawings for animations. We are having a lot of complaints about it. There is something with the jsmol popup menu, which gets cut out by the notebook cell, and can't be used (thus making exports impossible)
https://github.com/jupyter/notebook/issues/1872 discussing this has been opened in 2016 (pity that it was not resolved during ODK).
comment:46 Changed 2 months ago by
Indeed; too bad this was not brought up. That being said, we still are in good contact with the Jupyter team :) So if we provide them with a specific and easy to reproduce example (e.g. through a binder link) on the above issue, there are chances that we can make progress. It would be worth testing how things are in jupyterlab too.
comment:47 followup: ↓ 48 Changed 2 months ago by
By the way, I note that only jsmol can make stereographic plots for 3d viewing, though I don't know how often this capability is used. (People like it when advertising Sage!)
comment:48 in reply to: ↑ 47 Changed 2 months ago by
Replying to kcrisman:
By the way, I note that only jsmol can make stereographic plots for 3d viewing, though I don't know how often this capability is used. (People like it when advertising Sage!)
can't plotly do it? see e.g. https://plot.ly/~empet/13037.embed
comment:49 Changed 2 months ago by
By "only jsmol", I meant the only thing included standard in Sage. What is plotly? (Am I so behind that I missed a major new Sage internal plot mechanism? Yikes.)
comment:50 Changed 2 months ago by
I read your comments as "the only Python library that can do this sort if plotting"
plotly is Python library, no idea if it can work in Sage.
comment:51 Changed 4 weeks ago by
 Milestone changed from sage8.9 to sage9.1
Ticket retargeted after milestone closed
Some issues I'm aware of related to features that are only in sagenb:
<obj>?
) in the Jupyter notebook does not display HTMLformatted docs (esp. with latex formatting), while sagenb does. I believe Odile might be looking into this now. Not a significantly difficult issue but not trivial either.search_doc
does not work in the Jupyter notebookautomatic_names
only works in sagenb. This was also discussed recently on sagedevel: https://groups.google.com/d/msg/sagedevel/RoBZ36i1xo4/e6ZunaKRAwAJ I believeautomatic_names()
can be reimplemented in such a way that's mostly independent on the frontend, though one would still need frontend specific hooks for it to work. I haven't tried this yet though.I think at a minimum these issues should be fixed before taking further steps at abandoning the old sagenb.