#25837 closed task (duplicate)
Make the sage notebook optional
Reported by:  Timo Kaufmann  Owned by:  

Priority:  major  Milestone:  sageduplicate/invalid/wontfix 
Component:  packages: standard  Keywords:  notebook, sagenb 
Cc:  Erik Bray, Volker Braun, Jeroen Demeyer, KarlDieter Crisman, François Bissey, Antonio Rojas, Timo Kaufmann, Ximin Luo, Andrey Novoseltsev, Paulo César Pereira de Andrade, Nicolas M. Thiéry, Samuel Lelièvre, Julian Rüth, Harald Schilly, Steven Trogdon, Tobias Hansen, Thierry Monteil, William Stein  Merged in:  
Authors:  Reviewers:  Dima Pasechnik, John Palmieri  
Report Upstream:  N/A  Work issues:  
Branch:  u/chapoton/25837 (Commits, GitHub, GitLab)  Commit:  ec7931a58d5ffec3b08ea8274911c4e17e6a80f1 
Dependencies:  Stopgaps: 
Description (last modified by )
If you're coming from the deprecation notice in the notebook
The legacy SageNB notebook has been effectively deprecated for quite some time. We are now making that official. But that does not mean that Sage will not have notebook functionality. In fact, most of the functionality of the sage notebook can already be replaced by using the Juypter Notebook instead. For an introduction on how to use the new notebook and convert existing notebooks, see the documentation.
There are *some* features that are not yet ported to the new notebook however. We will not completely stop supporting the legacy notebook until those issues are resolved. Those features are listed below. If there is some feature you're missing that is not listed or you want to give general feedback, feel free to leave a comment on this ticket. If you do not have an account yet, you can log in with a GitHub account.
Implementation details
Reasons to make sagenb optional
 most of its functionality has been replaced for a while by Jupyter
 it is not actively maintained  merely on life support
 it has various open issues (looking at the "Critical" label in the github repo)
 unmaintained dependencies (https://github.com/sagemath/sagenb/issues/440#issuecomment381583952)
 incompatibility with the latest flaskbabel (https://github.com/sagemath/sagenb/issues/437)
 incompatibility with python3 (https://github.com/sagemath/sagenb/issues/343)
 an XSS vulnerability, unfixed for 3.5 years (https://github.com/sagemath/sagenb/issues/319)
 and for me personally: I don't want to package it. I feel like it causes a disproportionate amount of issues considering its relatively low value in today's Sage.
Reasons to keep sagenb around
 there are still some features depending on it
 there might still be people using it. An example would be @fbissey's university: https://trac.sagemath.org/ticket/25382#comment:16
In making it optional, we still make it possible to use it for now. And when it inevitably breaks further, the people relying on it can weigh the effort of keeping it alive themselves against the effort of switching to the new notebook.
Tickets and issues tracking progress/going in that direction
Necessary to keep all tested behaviour intact without sagenb ("pass the doctests")
 #30540: port/remove the graph editor
 port/remove the graph database navigator
 port/remove the interactive debugger
 #25382: do not include sagenb docs in sage
Misc changes
 #25852: add a deprecation warning to the sage notebook
 #25837: actually make sagenb optional
Functionality sagenb currently does better than Jupyter
 #25015: object introspection (
<obj>?
) in the Jupyter notebook does not display HTMLformatted docs (esp. with latex formatting), while sagenb does
 #24784:
search_doc
does not work in the Jupyter notebook
 #9048:
automatic_names
only works in sagenb
 Jupyter's converting to PDF "kind of sucks" (see #23330)
 #26101: With Jupyter,
view(...)
currently displays the pdf in an external pdf viewer, launched from the server. An inline view as in sagenb is nicer and would work remotely as well.
 Jupyter does not scroll large output (e.g. matrices) well; see this post on Stackoverflow
 probably supports matrix input, which does work in Sage cell server but not in Jupyter as of this Sage support discussion
Otherwise relevant
 #22431: first step in making sagenb python3 compatible
 https://wiki.sagemath.org/IpythonNotebook (which features of the Sage notebook are missing in the jupyter notebook, to be updated)
 MathJax? fonts  I don't know if it was possible to change the font in the legacy notebook either, but it's not easy to configure a different font in Jupyter notebook. See e.g.:
Change History (55)
comment:1 followup: 3 Changed 4 years ago by
comment:2 Changed 4 years ago by
I would also say that something "official" when one uses sagenb even when directly that says it is deprecated is necessary.
comment:3 Changed 4 years ago by
Description:  modified (diff) 

comment:4 Changed 4 years ago by
I modified the ticket description to incorporate (read: shamelessly copy) your additions.
It would be nice if we could get the deprecation warning into 8.3 so that users will have at least one release to prepare. Thats a very optimistic estimate of course, but getting it in 8.3 would be nice either way.
comment:5 Changed 4 years ago by
Description:  modified (diff) 

I have opened #25852 for that. I have only ever actually started the notebook once. Do you have an idea where best to add such a message? Just print it on the command line?
comment:6 Changed 4 years ago by
Probably on the notebook itself as well, which would require changes in sagenb.
comment:7 Changed 4 years ago by
Cc:  Antonio Rojas Timo Kaufmann Ximin Luo Andrey Novoseltsev Paulo César Pereira de Andrade Nicolas M. Thiéry Samuel Lelièvre Julian Rüth Harald Schilly Steven Trogdon Tobias Hansen Thierry Monteil William Stein added 

Keywords:  sagenb added 
cc a few people who package Sage in various ways
comment:8 Changed 4 years ago by
Another issue that was brought up here: https://ask.sagemath.org/question/43051/filedownloadaslatex/
Converting a Jupyter notebook to PDF kind of sucks at the moment for notebooks that contain a lot of math in latex. There are two main cases in the notebook where latex can appear, and both have issues in sage:
1) If an object output from a code cell has _render_latex_
then the notebook will display the rendered latex with mathjax. However, when converting the notebook to PDF, nbconvert just takes the plaintext representation of the output, not the latex, and puts it in a verbatim block. Not good, especially not for Sage.
2) If a markdown cell contains equations in latex, nbconvert *will* render them correctly, but only if you have pandoc installed. This is already a known issue, but since pandoc does not come with Sage and is not a dependency this makes for a sucky user experience for something that should Just Work.
I'm not sure what to do about the first case; it seems like an issue for nbconvert but I haven't explored it too deeply. For the latter case (which probably also affects the first case) I wonder if we shouldn't just make pandoc another sage dependency.
comment:9 Changed 4 years ago by
Description:  modified (diff) 

I've updated the ticket. The flask issue was resolved (thanks @fchapoton!), I added the pdf complaint and improved formatting.
comment:10 Changed 4 years ago by
Description:  modified (diff) 

comment:11 Changed 4 years ago by
Description:  modified (diff) 

I've added a section intended for end users. Please improve on that if you have more knowledge about sagenb or the Jupyter Notebook, which you probably do.
comment:12 Changed 4 years ago by
Description:  modified (diff) 

comment:13 Changed 4 years ago by
Description:  modified (diff) 

comment:14 Changed 4 years ago by
Description:  modified (diff) 

comment:15 followup: 16 Changed 4 years ago by
Those features are listed below.
Where is the list? I can't find it.
I touched on this on page 26 of a talk I gave yesterday (https://share.cocalc.com/share/6cd832d3c52341e39e54c8f2d2e8fa2a/travel/2018/201807icms/talks/cocalc/slides/slides.html#1)
One example is that if you start running:
for i in range(10): print(i) sys.stdout.flush() time.sleep(1)
then close and open your browser (to simulate a network failure, say), you should not loose output. In Jupyter, you do.
comment:16 Changed 4 years ago by
Replying to was:
Those features are listed below.
Where is the list? I can't find it.
It's in the description of the ticket, but perhaps that still needs more reformatting, because there are a number of thing "below" that sentence that include, but are not limited to, missing features.
Most of it falls under the heading "Functionality sagenb currently does better than Jupyter" though that list is far from complete. There's also a link to a wiki page listing other issues.
comment:17 Changed 4 years ago by
Description:  modified (diff) 

comment:18 Changed 4 years ago by
Description:  modified (diff) 

comment:19 Changed 4 years ago by
"#24784: search_doc does not work in the Jupyter notebook" is now ready for review.
comment:20 followup: 21 Changed 4 years ago by
Could someone (having the required rights) please now merge the 4 top pull requests in sagenb github, including the one related to the present ticket ?
comment:21 Changed 4 years ago by
comment:22 Changed 4 years ago by
Branch:  → u/chapoton/25837 

Commit:  → ec7931a58d5ffec3b08ea8274911c4e17e6a80f1 
New commits:
ec7931a  make sagenb optional

comment:23 Changed 4 years ago by
Description:  modified (diff) 

comment:24 Changed 4 years ago by
Milestone:  sage8.3 → sage8.8 

comment:25 Changed 4 years ago by
remaining imports in 8.7.b6 :
git grep "import.*sagenb" src/ src/doc/en/developer/sagenb/index.rst:The most important section involves how to update your new sagenb src/sage/all.py:lazy_import('sagenb.notebook.notebook_object', 'notebook') src/sage/all.py:lazy_import('sagenb.notebook.notebook_object', 'inotebook') src/sage/all.py:lazy_import('sagenb.notebook.sage_email', 'email') src/sage/all.py:lazy_import('sagenb.notebook.interact', 'interact') src/sage/graphs/graph_editor.py: import sagenb.notebook.interact src/sage/interacts/library.py: import sagenb.notebook.interact
comment:26 followup: 29 Changed 4 years ago by
Apparently this question implies that we should add "can't do the table command" in Jupyter. Is that accurate?
comment:28 Changed 4 years ago by
Status:  new → needs_review 

comment:29 Changed 4 years ago by
Replying to kcrisman:
Apparently this question implies that we should add "can't do the table command" in Jupyter. Is that accurate?
Now tracked at #27656.
comment:30 followup: 31 Changed 4 years ago by
Reviewers:  → Dima Pasechnik 

Status:  needs_review → positive_review 
Does anything stop this from going forward?
comment:31 Changed 4 years ago by
Does anything stop this from going forward?
Well, does Jupyter actually do all this stuff yet or not?
Also, for people who download a new Sage (which many do, as opposed to pulling and upgrading), it's not clear to me what happens when they try to use the notebook export functionality (which should still be the default as of now). How do they acquire this optional package if they have a binaryonly Sage? Or would the export still accurately export to Jupyter anyway?
comment:32 followup: 35 Changed 4 years ago by
Status:  positive_review → needs_info 

IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.
Is there a reason to rush this forward? If so, what tickets could use my help?
comment:33 Changed 4 years ago by
Description:  modified (diff) 

comment:34 Changed 3 years ago by
Milestone:  sage8.8 → sage8.9 

Tickets still needing working or clarification should be moved to the next release milestone at the soonest (please feel free to revert if you think the ticket is close to being resolved).
comment:35 followup: 37 Changed 3 years ago by
Replying to embray:
IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.
Is there a reason to rush this forward? If so, what tickets could use my help?
If SageNB doesn't work with Python 3, then we will need to drop it when we make the switch. Since the switch is coming up, we should officially deprecate this as soon as possible. If we manage to switch to Python 3 by January 1, we already won't make the required (by Sage policy) 6 months of deprecation time.
comment:36 Changed 3 years ago by
Status:  needs_info → needs_work 

SageNB won't be ported to Python 3, it's been clear for ages I think.
I think this patch is potentially breaking the current notebooklaunching setup,
as we don't want to have a not always installed package pointed to by
Run the old Sage Notebook
menu item.
The latter should not be shown if the package is not installed. In this case instead a message saying something like "Install sagenb optional package for the old SageNB" should be there.
comment:37 Changed 3 years ago by
Replying to jhpalmieri:
Replying to embray:
IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.
Is there a reason to rush this forward? If so, what tickets could use my help?
If SageNB doesn't work with Python 3, then we will need to drop it when we make the switch. Since the switch is coming up, we should officially deprecate this as soon as possible. If we manage to switch to Python 3 by January 1, we already won't make the required (by Sage policy) 6 months of deprecation time.
I added a deprecation warning close to a year ago: https://trac.sagemath.org/ticket/25852
So I think we're good on that front if we decide to drop it with the first python3 release. Feature parity is a different issue though.
comment:38 Changed 3 years ago by
I have made a pull request to change the front page of the exporter.
comment:40 Changed 3 years ago by
We will not completely stop supporting the legacy notebook until those issues are resolved.
Did those issues get resolved, and/or will they soon? automatic_names
for instance would be an actual regression. Does Jupyter now support matrix input for interacts? Just wondering, before the switch is actually pulled ...
comment:41 Changed 3 years ago by
And was the graph stuff ever removed as is suggested in the description?
comment:42 Changed 3 years ago by
When we talk about regressions, I think the context is: we are imagining people converting their notebooks from SageNB to Jupyter and not using SageNB any more (as opposed to: using SageNB for some tasks, Jupyter for others). In that context, it is also worthwhile to ask what features Jupyter has that SageNB doesn't. I don't think we are just losing things by moving from SageNB to Jupyter; aside from Python 3 compatibility, I think we gain some things by moving to Jupyter. So it is a tradeoff. At what point does the balance tip to the Jupyter side? Maybe the Python 3 support is enough on its own, in fact.
comment:43 Changed 3 years ago by
Maybe the Python 3 support is enough on its own, in fact.
Obviously that is the consensus decision. I'm just quoting this ticket right now when I mention these things, and would hate to have them disappear for all time. But as you said elsewhere, someone has to work on them.
comment:44 Changed 3 years ago by
A functionality that Jupyter is currently missing over sagenb is the rich text editor. This may possibly be resolved by:
comment:45 Changed 3 years ago by
the most pressing issue is inability to normally use jsmol in Jupyter notebooks, needed to export 3d drawings for animations. We are having a lot of complaints about it. There is something with the jsmol popup menu, which gets cut out by the notebook cell, and can't be used (thus making exports impossible)
https://github.com/jupyter/notebook/issues/1872 discussing this has been opened in 2016 (pity that it was not resolved during ODK).
comment:46 Changed 3 years ago by
Indeed; too bad this was not brought up. That being said, we still are in good contact with the Jupyter team :) So if we provide them with a specific and easy to reproduce example (e.g. through a binder link) on the above issue, there are chances that we can make progress. It would be worth testing how things are in jupyterlab too.
comment:47 followup: 48 Changed 3 years ago by
By the way, I note that only jsmol can make stereographic plots for 3d viewing, though I don't know how often this capability is used. (People like it when advertising Sage!)
comment:48 Changed 3 years ago by
Replying to kcrisman:
By the way, I note that only jsmol can make stereographic plots for 3d viewing, though I don't know how often this capability is used. (People like it when advertising Sage!)
can't plotly do it? see e.g. https://plot.ly/~empet/13037.embed
comment:49 Changed 3 years ago by
By "only jsmol", I meant the only thing included standard in Sage. What is plotly? (Am I so behind that I missed a major new Sage internal plot mechanism? Yikes.)
comment:50 Changed 3 years ago by
I read your comments as "the only Python library that can do this sort if plotting"
plotly is Python library, no idea if it can work in Sage.
comment:51 Changed 3 years ago by
Milestone:  sage8.9 → sage9.1 

Ticket retargeted after milestone closed
comment:52 Changed 3 years ago by
Milestone:  sage9.1 → sageduplicate/invalid/wontfix 

Status:  needs_work → needs_review 
comment:53 Changed 3 years ago by
Reviewers:  Dima Pasechnik → Dima Pasechnik, John Palmieri 

Status:  needs_review → positive_review 
comment:54 Changed 3 years ago by
Resolution:  → duplicate 

Status:  positive_review → closed 
comment:55 Changed 2 years ago by
Description:  modified (diff) 

Some issues I'm aware of related to features that are only in sagenb:
<obj>?
) in the Jupyter notebook does not display HTMLformatted docs (esp. with latex formatting), while sagenb does. I believe Odile might be looking into this now. Not a significantly difficult issue but not trivial either.search_doc
does not work in the Jupyter notebookautomatic_names
only works in sagenb. This was also discussed recently on sagedevel: https://groups.google.com/d/msg/sagedevel/RoBZ36i1xo4/e6ZunaKRAwAJ I believeautomatic_names()
can be reimplemented in such a way that's mostly independent on the frontend, though one would still need frontend specific hooks for it to work. I haven't tried this yet though.I think at a minimum these issues should be fixed before taking further steps at abandoning the old sagenb.