Opened 16 months ago

Last modified 10 days ago

#25837 needs_work task

Make the sage notebook optional

Reported by: gh-timokau Owned by:
Priority: major Milestone: sage-8.9
Component: packages: standard Keywords: notebook, sagenb
Cc: embray, vbraun, jdemeyer, kcrisman, fbissey, arojas, gh-timokau, infinity0, novoselt, pcpa, nthiery, slelievre, saraedum, schilly, strogdon, thansen, tmonteil, was Merged in:
Authors: Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: u/chapoton/25837 (Commits) Commit: ec7931a58d5ffec3b08ea8274911c4e17e6a80f1
Dependencies: Stopgaps:

Description (last modified by kcrisman)

If you're coming from the deprecation notice in the notebook

The legacy sage notebook has been effectively deprecated for quite some time. We are now making that official. But that does not mean that sage will not have notebook functionality. In fact, most of the functionality of the sage notebook can already be replaced by using the Juypter Notebook instead. For an introduction on how to use the new notebook and convert existing notebooks, see the documentation.

There are *some* features that are not yet ported to the new notebook hovever. We will not completely stop supporting the legacy notebook until those issues are resolved. Those features are listed below. If there is some feature you're missing that is not listed or you want to give general feedback, feel free to leave a comment on this ticket. If you do not have an account yet, you can log in with a GitHub account.

Implementation details

Reasons to make sagenb optional

  • most of its functionality has been replaced for a while by Jupyter
  • it is not actively maintained -- merely on life support
  • and for me personally: I don't want to package it. I feel like it causes an unproportional amount of issues considering its relatively low value in todays sage.

Reasons to keep sagenb around

  • there are still some features depending on it

In making it optional, we still make it possible to use it for now. An when it inevitably breaks further, the people relying on it can weigh the effort of keeping it alive themselves against the effort of switching to the new notebook.

Tickets and issues tracking progress/going in that direction

Necessary to keep all tested behaviour in tact without sagenb ("pass the doctests")

  • port/remove the graph editor
  • port/remove the graph database navigator
  • port/remove the interactive debugger
  • #25382: do not include sagenb docs in sage

Misc changes

  • #25852: add a deprecation warning to the sage notebook
  • actually make sagenb optional

Functionality sagenb currently does better than Jupyter

  • #25015: object introspection (<obj>?) in the Jupyter notebook does not display HTML-formatted docs (esp. with latex formatting), while sagenb does
  • #24784: search_doc does not work in the Jupyter notebook
  • #9048: automatic_names only works in sagenb
  • #26101: With Jupyter, view(...) currently displays the pdf in an external pdf viewer, launched from the server. An inline view as in sagenb is nicer and would work remotely as well.

Otherwise relevant

Change History (43)

comment:1 follow-up: Changed 16 months ago by embray

Some issues I'm aware of related to features that are only in sagenb:

  • #25015 - object introspection (<obj>?) in the Jupyter notebook does not display HTML-formatted docs (esp. with latex formatting), while sagenb does. I believe Odile might be looking into this now. Not a significantly difficult issue but not trivial either.
  • #24784 - search_doc does not work in the Jupyter notebook

I think at a minimum these issues should be fixed before taking further steps at abandoning the old sagenb.

comment:2 Changed 16 months ago by kcrisman

I would also say that something "official" when one uses sagenb even when directly that says it is deprecated is necessary.

comment:3 in reply to: ↑ 1 Changed 16 months ago by gh-timokau

  • Description modified (diff)

comment:4 Changed 16 months ago by gh-timokau

I modified the ticket description to incorporate (read: shamelessly copy) your additions.

It would be nice if we could get the deprecation warning into 8.3 so that users will have at least one release to prepare. Thats a very optimistic estimate of course, but getting it in 8.3 would be nice either way.

comment:5 Changed 16 months ago by gh-timokau

  • Description modified (diff)

I have opened #25852 for that. I have only ever actually started the notebook once. Do you have an idea where best to add such a message? Just print it on the command line?

comment:6 Changed 16 months ago by kcrisman

Probably on the notebook itself as well, which would require changes in sagenb.

comment:7 Changed 16 months ago by slelievre

  • Cc arojas gh-timokau infinity0 novoselt pcpa nthiery slelievre saraedum schilly strogdon thansen tmonteil was added
  • Keywords sagenb added

cc a few people who package Sage in various ways

comment:8 Changed 16 months ago by embray

Another issue that was brought up here: https://ask.sagemath.org/question/43051/file-download-as-latex/

Converting a Jupyter notebook to PDF kind of sucks at the moment for notebooks that contain a lot of math in latex. There are two main cases in the notebook where latex can appear, and both have issues in sage:

1) If an object output from a code cell has _render_latex_ then the notebook will display the rendered latex with mathjax. However, when converting the notebook to PDF, nbconvert just takes the plain-text representation of the output, not the latex, and puts it in a verbatim block. Not good, especially not for Sage.

2) If a markdown cell contains equations in latex, nbconvert *will* render them correctly, but only if you have pandoc installed. This is already a known issue, but since pandoc does not come with Sage and is not a dependency this makes for a sucky user experience for something that should Just Work.

I'm not sure what to do about the first case; it seems like an issue for nbconvert but I haven't explored it too deeply. For the latter case (which probably also affects the first case) I wonder if we shouldn't just make pandoc another sage dependency.

comment:9 Changed 16 months ago by gh-timokau

  • Description modified (diff)

I've updated the ticket. The flask issue was resolved (thanks @fchapoton!), I added the pdf complaint and improved formatting.

comment:10 Changed 16 months ago by jdemeyer

  • Description modified (diff)

comment:11 Changed 16 months ago by gh-timokau

  • Description modified (diff)

I've added a section intended for end users. Please improve on that if you have more knowledge about sagenb or the Jupyter Notebook, which you probably do.

comment:12 Changed 16 months ago by tmonteil

  • Description modified (diff)

comment:13 Changed 16 months ago by egourgoulhon

  • Description modified (diff)

comment:14 Changed 16 months ago by embray

  • Description modified (diff)

comment:15 follow-up: Changed 16 months ago by was

Those features are listed below.

Where is the list? I can't find it.

I touched on this on page 26 of a talk I gave yesterday (https://share.cocalc.com/share/6cd832d3-c523-41e3-9e54-c8f2d2e8fa2a/travel/2018/2018-07-icms/talks/cocalc/slides/slides.html#1)

One example is that if you start running:

for i in range(10):
   print(i)
   sys.stdout.flush()
   time.sleep(1)

then close and open your browser (to simulate a network failure, say), you should not loose output. In Jupyter, you do.

Last edited 16 months ago by was (previous) (diff)

comment:16 in reply to: ↑ 15 Changed 16 months ago by embray

Replying to was:

Those features are listed below.

Where is the list? I can't find it.

It's in the description of the ticket, but perhaps that still needs more reformatting, because there are a number of thing "below" that sentence that include, but are not limited to, missing features.

Most of it falls under the heading "Functionality sagenb currently does better than Jupyter" though that list is far from complete. There's also a link to a wiki page listing other issues.

comment:17 Changed 15 months ago by nthiery

  • Description modified (diff)

comment:18 Changed 15 months ago by nthiery

  • Description modified (diff)

comment:19 Changed 15 months ago by jhpalmieri

"#24784: search_doc does not work in the Jupyter notebook" is now ready for review.

comment:20 follow-up: Changed 15 months ago by chapoton

Could someone (having the required rights) please now merge the 4 top pull requests in sagenb github, including the one related to the present ticket ?

https://github.com/sagemath/sagenb/pulls

comment:21 in reply to: ↑ 20 Changed 13 months ago by dimpase

Replying to chapoton:

Could someone (having the required rights) please now merge the 4 top pull requests in sagenb github, including the one related to the present ticket ?

https://github.com/sagemath/sagenb/pulls

done, I guess. See #26499 for the new release of sagenb.

comment:22 Changed 11 months ago by chapoton

  • Branch set to u/chapoton/25837
  • Commit set to ec7931a58d5ffec3b08ea8274911c4e17e6a80f1

New commits:

ec7931amake sagenb optional

comment:23 Changed 9 months ago by kcrisman

  • Description modified (diff)

comment:24 Changed 9 months ago by chapoton

  • Milestone changed from sage-8.3 to sage-8.8

comment:25 Changed 9 months ago by chapoton

remaining imports in 8.7.b6 :

git grep "import.*sagenb" src/
src/doc/en/developer/sagenb/index.rst:The most important section involves how to update your new sagenb
src/sage/all.py:lazy_import('sagenb.notebook.notebook_object', 'notebook')
src/sage/all.py:lazy_import('sagenb.notebook.notebook_object', 'inotebook')
src/sage/all.py:lazy_import('sagenb.notebook.sage_email', 'email')
src/sage/all.py:lazy_import('sagenb.notebook.interact', 'interact')
src/sage/graphs/graph_editor.py:    import sagenb.notebook.interact
src/sage/interacts/library.py:        import sagenb.notebook.interact

comment:26 follow-up: Changed 7 months ago by kcrisman

Apparently this question implies that we should add "can't do the table command" in Jupyter. Is that accurate?

comment:27 Changed 7 months ago by jhpalmieri

It works for me.

comment:28 Changed 7 months ago by dimpase

  • Status changed from new to needs_review

comment:29 in reply to: ↑ 26 Changed 7 months ago by slelievre

Replying to kcrisman:

Apparently this question implies that we should add "can't do the table command" in Jupyter. Is that accurate?

Now tracked at #27656.

comment:30 follow-up: Changed 7 months ago by dimpase

  • Reviewers set to Dima Pasechnik
  • Status changed from needs_review to positive_review

Does anything stop this from going forward?

comment:31 in reply to: ↑ 30 Changed 7 months ago by kcrisman

Does anything stop this from going forward?

Well, does Jupyter actually do all this stuff yet or not?

Also, for people who download a new Sage (which many do, as opposed to pulling and upgrading), it's not clear to me what happens when they try to use the notebook export functionality (which should still be the default as of now). How do they acquire this optional package if they have a binary-only Sage? Or would the export still accurately export to Jupyter anyway?

comment:32 follow-up: Changed 7 months ago by embray

  • Status changed from positive_review to needs_info

IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.

Is there a reason to rush this forward? If so, what tickets could use my help?

comment:33 Changed 7 months ago by kcrisman

  • Description modified (diff)

comment:34 Changed 5 months ago by embray

  • Milestone changed from sage-8.8 to sage-8.9

Tickets still needing working or clarification should be moved to the next release milestone at the soonest (please feel free to revert if you think the ticket is close to being resolved).

comment:35 in reply to: ↑ 32 ; follow-up: Changed 2 months ago by jhpalmieri

Replying to embray:

IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.

Is there a reason to rush this forward? If so, what tickets could use my help?

If SageNB doesn't work with Python 3, then we will need to drop it when we make the switch. Since the switch is coming up, we should officially deprecate this as soon as possible. If we manage to switch to Python 3 by January 1, we already won't make the required (by Sage policy) 6 months of deprecation time.

comment:36 Changed 2 months ago by dimpase

  • Status changed from needs_info to needs_work

SageNB won't be ported to Python 3, it's been clear for ages I think.

I think this patch is potentially breaking the current notebook-launching setup, as we don't want to have a not always installed package pointed to by Run the old Sage Notebook menu item.

The latter should not be shown if the package is not installed. In this case instead a message saying something like "Install sagenb optional package for the old SageNB" should be there.

Last edited 2 months ago by dimpase (previous) (diff)

comment:37 in reply to: ↑ 35 Changed 2 months ago by gh-timokau

Replying to jhpalmieri:

Replying to embray:

IMO there shouldn't be any rush on this until at a minimum the issues listed in the description of this ticket are dealt with.

Is there a reason to rush this forward? If so, what tickets could use my help?

If SageNB doesn't work with Python 3, then we will need to drop it when we make the switch. Since the switch is coming up, we should officially deprecate this as soon as possible. If we manage to switch to Python 3 by January 1, we already won't make the required (by Sage policy) 6 months of deprecation time.

I added a deprecation warning close to a year ago: https://trac.sagemath.org/ticket/25852

So I think we're good on that front if we decide to drop it with the first python3 release. Feature parity is a different issue though.

comment:38 Changed 8 weeks ago by chapoton

I have made a pull request to change the front page of the exporter.

https://github.com/vbraun/ExportSageNB/pull/16

comment:39 Changed 6 weeks ago by chapoton

This pull request have been merged.

comment:40 Changed 11 days ago by kcrisman

We will not completely stop supporting the legacy notebook until those issues are resolved.

Did those issues get resolved, and/or will they soon? automatic_names for instance would be an actual regression. Does Jupyter now support matrix input for interacts? Just wondering, before the switch is actually pulled ...

comment:41 Changed 11 days ago by kcrisman

And was the graph stuff ever removed as is suggested in the description?

comment:42 Changed 11 days ago by jhpalmieri

When we talk about regressions, I think the context is: we are imagining people converting their notebooks from SageNB to Jupyter and not using SageNB any more (as opposed to: using SageNB for some tasks, Jupyter for others). In that context, it is also worthwhile to ask what features Jupyter has that SageNB doesn't. I don't think we are just losing things by moving from SageNB to Jupyter; aside from Python 3 compatibility, I think we gain some things by moving to Jupyter. So it is a trade-off. At what point does the balance tip to the Jupyter side? Maybe the Python 3 support is enough on its own, in fact.

comment:43 Changed 10 days ago by kcrisman

Maybe the Python 3 support is enough on its own, in fact.

Obviously that is the consensus decision. I'm just quoting this ticket right now when I mention these things, and would hate to have them disappear for all time. But as you said elsewhere, someone has to work on them.

Note: See TracTickets for help on using tickets.