Opened 10 months ago

Closed 9 months ago

Last modified 7 months ago

#30246 closed enhancement (fixed)

Add optional packages jupyterlab, jupyterlab_widgets to support interacts in JupyterLab

Reported by: slelievre Owned by:
Priority: critical Milestone: sage-9.2
Component: interact Keywords: jupyterlab
Cc: slelievre Merged in:
Authors: Joshua Campbell Reviewers: Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: e169060 (Commits, GitHub, GitLab) Commit:
Dependencies: Stopgaps:

Status badges

Description

Before this ticket: interacts don't work in JupyterLab.

Reported at:

Related:

  • #29728: Meta-ticket: improve compatibility with the Python library

Change History (34)

comment:1 follow-up: Changed 9 months ago by mkoeppe

Could this just be a documentation issue? Is it documented somewhere what extensions need to be installed in jupyterlab so that sage works?

comment:2 Changed 9 months ago by mkoeppe

  • Dependencies set to #26059

comment:3 in reply to: ↑ 1 Changed 9 months ago by gh-jcamp0x2a

Replying to mkoeppe:

Could this just be a documentation issue? Is it documented somewhere what extensions need to be installed in jupyterlab so that sage works?

I was able to get interacts working in jupyterlab 2.2.5 using the following command: ipywidgets install instructions

./sage --jupyter labextension install @jupyter-widgets/jupyterlab-manager@2.0

This requires nodejs >= 10 to be installed though. Looks like jupyterlab's extension ecosystem is all based on npm. Grabbing the extension tarball directly, still need node for bundling. (webpack, if I remember; been awhile since I've done anything with node.)

So unless we want to make nodejs an optional->standard package as well, maybe just leave it to the user to install and make a note in the documentation regarding interact functionality? Similar to how imagemagick and ffmpeg are noted in animated plots.

comment:4 follow-up: Changed 9 months ago by mkoeppe

Is installation via https://pypi.org/project/nodeenv/ an option?

comment:5 in reply to: ↑ 4 Changed 9 months ago by gh-jcamp0x2a

Replying to mkoeppe:

Is installation via https://pypi.org/project/nodeenv/ an option?

Cool stuff. I was indeed able to get interacts working without a system nodejs using ./sage --pip install nodeenv and creating a nodeenv to do the jupyter-widgets extension installation. Since node isn't needed for running jupyterlab, that nodeenv could probably get wiped out automatically following installation.

Not having node though does prevent you from browsing/installing other extensions in the extensions sidebar of jupyterlab. I played around with modifying the main sage script to activate the nodeenv when using ./sage -n jupyterlab and was able to install, for example, the jupyterlab-toc extension and use it without issue.

I'll continue experimenting with this for a bit longer, do some tidying up, and push a branch for review.

comment:6 Changed 9 months ago by gh-jcamp0x2a

  • Authors set to Joshua Campbell
  • Branch set to u/gh-jcamp0x2a/30246
  • Commit set to d2d8c4054e946a93fa8bf3bbfe8865eeb2b6f2fa
  • Status changed from new to needs_review

I've pushed a branch for review. It adds 4 new optional packages: nodeenv, nodejs, jupyterlab, and jupyterlab_widgets. The last one, jupyterlab_widgets, gets you everything.

I opted to break them up in this way for (1) convenience in packaging them and (2) flexibility in installing them. For example, if you just want the JupyterLab client and don't care about interacts or other extensions or if you don't want JupyterLab but would like nodejs present in your Sage environment for some other purpose. Please let me know if you'd prefer a different structuring.

Shell scripting isn't my forte, so I welcome any feedback regarding coding conventions and best practices!

comment:7 follow-up: Changed 9 months ago by gh-jcamp0x2a

I suppose this also includes what should've been worked under #26059 first and separately (just the jupyterlab package). I apologize for that; got carried away :)

comment:8 in reply to: ↑ 7 Changed 9 months ago by slelievre

  • Dependencies #26059 deleted

Replying to gh-jcamp0x2a:

includes what should've been worked under #26059 first and separately

We can mark the other ticket as solved by this one.

I would use https in the home page url for JupyterLab.

comment:9 Changed 9 months ago by git

  • Commit changed from d2d8c4054e946a93fa8bf3bbfe8865eeb2b6f2fa to b7010f68662a73cb26d16216b430751173fd6ae3

Branch pushed to git repo; I updated commit sha1. New commits:

c0b02c4Fix sage-sh/-buildsh prompt getting wiped out by nodeenv
b7010f6Use https url in doc for jupyterlab package

comment:10 Changed 9 months ago by mkoeppe

This looks great overall.

I'm getting the following with this branch (without installing any packages):

./sage -h
/Users/mkoeppe/s/sage/sage-rebasing/src/bin/sage-env: line 69: cd: /Users/mkoeppe/s/sage/sage-rebasing/local/share/nodejs/: No such file or directory
SageMath version 9.2.beta10, Release Date: 2020-08-23
...
  --notebook=[...]    -- start the Sage notebook (valid options are
                         'default', 'jupyter', and 'export')
                         Current default is 'export' from sagenb to jupyter
  -n, --notebook      -- shortcut for --notebook=default
...
$ ./sage -n jupyterlab
/Users/mkoeppe/s/sage/sage-rebasing/src/bin/sage-env: line 69: cd: /Users/mkoeppe/s/sage/sage-rebasing/local/share/nodejs/: No such file or directory
Jupyterlab is not installed (at least not in this Sage installation).
You can install it by running
  sage --pip install jupyterlab

comment:11 Changed 9 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:12 Changed 9 months ago by git

  • Commit changed from b7010f68662a73cb26d16216b430751173fd6ae3 to 4770d3b56288e230c9a94139c3373bb286a6ecc9

Branch pushed to git repo; I updated commit sha1. New commits:

4770d3bResolve nodeenv_activate symlink _after_ checking that it exists

comment:13 Changed 9 months ago by git

  • Commit changed from 4770d3b56288e230c9a94139c3373bb286a6ecc9 to 29359e1434f5464023c72bfe652118c2fbc1906a

Branch pushed to git repo; I updated commit sha1. New commits:

29359e1Instruct user to install jupyterlab using -i instead of --pip install

comment:14 follow-up: Changed 9 months ago by gh-jcamp0x2a

  • Status changed from needs_work to needs_review

Thank you for catching that!

I've pushed a couple more changes, the first making sure to check that nodeenv_activate symlink exists before trying to resolve it, and the second mentioning to the user to install using sage -i jupyterlab instead of sage --pip install jupyterlab.

Oh, do you happen to know of a good way to uninstall Sage packages? I tested these last couple changes by manually removing/moving files around in SAGE_LOCAL. I'd like to get back to a state without, for example, jupyterlab for testing, but I'd also like to avoid having to do a full clean and rebuild. I suppose I could be doing this in a VM and just snapshot and rollback.

comment:15 Changed 9 months ago by tmonteil

  • Status changed from needs_review to needs_work

This ticket breaks the patchbots (at least the ones i run). It seems due to some modification in src/bin/sage-env that tries to cd into a nonexistent local/share/nodejs directory. When the patchbots run:

./sage --python --version

They get:

/home/sagemath/sage-9.1/src/bin/sage-env: line 69: cd: /home/sagemath/sage-9.1/local/share/nodejs/: No such file or directory
Python 3.7.3

The error should be catched (e.g. stderr could be redirected to /dev/null).

comment:16 follow-up: Changed 9 months ago by tmonteil

  • Status changed from needs_work to needs_review

Apparently, you fixed the issue already, sorry for the noise, let us see if the patchbots are happy now.

comment:17 in reply to: ↑ 16 Changed 9 months ago by gh-jcamp0x2a

Replying to tmonteil:

Apparently, you fixed the issue already, sorry for the noise, let us see if the patchbots are happy now.

No worries. Thank you for running those patchbots.

comment:18 Changed 9 months ago by mkoeppe

  • Priority changed from major to critical

comment:19 in reply to: ↑ 14 Changed 9 months ago by mkoeppe

Replying to gh-jcamp0x2a:

mentioning to the user to install using sage -i jupyterlab instead of sage --pip install jupyterlab.

Would it perhaps be better to direct the user to do sage -i jupyterlab_widgets so that a full-featured jupyterlab is installed?

comment:20 Changed 9 months ago by mkoeppe

Also, could you update the help messages in src/bin/sage so that jupyterlab is mentioned as an option. (Also I would guess that the default that is mentioned there is not correct.)

comment:21 Changed 9 months ago by mkoeppe

on macOS:

[nodejs-12.18.3] usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[nodejs-12.18.3] 	[-e pattern] [-f file] [--binary-files=value] [--color=when]
[nodejs-12.18.3] 	[--context[=num]] [--directories=action] [--label] [--line-buffered]
[nodejs-12.18.3] 	[--null] [pattern] [file ...]
[nodejs-12.18.3] Error determining which nodejs version to install ... exiting

comment:22 follow-up: Changed 9 months ago by mkoeppe

grep -P is not portable

comment:23 Changed 9 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:24 in reply to: ↑ 22 Changed 9 months ago by gh-jcamp0x2a

Replying to mkoeppe:

Would it perhaps be better to direct the user to do sage -i jupyterlab_widgets so that a full-featured jupyterlab is installed?

I will update the message to suggest jupyterlab_widgets by default, with a short aside about a minimal installation using just jupyterlab.

Replying to mkoeppe:

Also, could you update the help messages in src/bin/sage so that jupyterlab is mentioned as an option. (Also I would guess that the default that is mentioned there is not correct.)

I will add mention of jupyterlab to the help messages. I'm not sure I understand you about the default. The help messages I see list jupyter as the default, and indeed running sage -n or sage -n default starts jupyter.

Replying to mkoeppe:

grep -P is not portable

I should know better. I remember this bit me on Solaris a few years ago, too. I will fix this.

comment:25 Changed 9 months ago by git

  • Commit changed from 29359e1434f5464023c72bfe652118c2fbc1906a to 0a06353ae7c91a8e66b6d1ead2561567c2b359b8

Branch pushed to git repo; I updated commit sha1. New commits:

3f1aad9Recommend jupyterlab_widgets, mention jupyterlab as minimal install
bcbccd6Mention jupyterlab in basic and advanced help messages
0a06353Don't use grep -P in nodejs spkg-install

comment:26 Changed 9 months ago by gh-jcamp0x2a

  • Status changed from needs_work to needs_review

Upon re-reading the basic help message, I saw what you mentioned about the default needing to be updated. I've pushed changes that I believe address your feedback items.

I also came across sage --notebook --help which had already mentioned jupyterlab and so didn't need updating, but I found that it also mentions an ipython option:

--notebook [NOTEBOOK], -n [NOTEBOOK], -notebook [NOTEBOOK]
                      The notebook to run [one of: default, ipython,
                      jupyter, jupyterlab, export]. Default is jupyter

Looking at the code in src/bin/sage-notebook, this option is currently just an alias for jupyter. Would it be valuable to mention this option in the basic and/or advanced help messages for sage as well?

comment:27 Changed 9 months ago by mkoeppe

  • Branch changed from u/gh-jcamp0x2a/30246 to u/mkoeppe/30246

comment:28 follow-up: Changed 9 months ago by mkoeppe

  • Commit changed from 0a06353ae7c91a8e66b6d1ead2561567c2b359b8 to e169060957d2aca9ba657599ca7f9fa4a3df23a1
  • Reviewers set to Matthias Koeppe
  • Status changed from needs_review to positive_review

This works for me on macOS and is looking great overall.

I have added dependencies to make sure that a parallel build works correctly.


New commits:

e169060build/pkgs/{jupyterlab_widgets,nodeenv}: Add dependencies

comment:29 Changed 9 months ago by mkoeppe

  • Summary changed from Support interacts in JupyterLab to Add optional packages jupyterlab, jupyterlab_widgets to support interacts in JupyterLab

comment:30 in reply to: ↑ 28 Changed 9 months ago by gh-jcamp0x2a

Replying to mkoeppe:

This works for me on macOS and is looking great overall.

I have added dependencies to make sure that a parallel build works correctly.

Awesome! Thank you for all your time and assistance on this ticket.

comment:31 Changed 9 months ago by vbraun

  • Branch changed from u/mkoeppe/30246 to e169060957d2aca9ba657599ca7f9fa4a3df23a1
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:32 Changed 8 months ago by charpent

  • Commit e169060957d2aca9ba657599ca7f9fa4a3df23a1 deleted

Minor booboo : after (successful) installlation, jupyterlab_widgets is still marked as not installed:

charpent@zen-book-flip:/usr/local/sage-exp$ ./sage -optional | grep jupyter
gap_jupyter.............................0.9 (0.9)
jupyterlab..............................2.2.6 (2.2.6)
jupyterlab_widgets......................none (2.0)
pari_jupyter............................1.3.2 (1.3.2)
r_jupyter...............................none (not_installed)
singular_jupyter........................0.9.7 (0.9.7)

Untidy, and potentially confusing...

comment:33 Changed 8 months ago by gh-jcamp0x2a

I believe the version in parentheses is the installed version while the other is the available version. I'm not sure why the available version isn't displayed since there is a package-version.txt present for jupyterlab_widgets, but I'm thinking this may be a problem for "script" packages in general. For example:

$ ./sage --installed | grep none
jupyterlab_widgets......................none (2.0)
nodejs..................................none (12.18.3)
sage_conf...............................none (none)
sagelib.................................none (9.2.beta10)
zope_interface..........................none (none)

comment:34 Changed 7 months ago by dimpase

Has anyone figured out how to use MathJax? in JupyterLab? markdown cells?

Note: See TracTickets for help on using tickets.