Opened 4 years ago

Closed 2 years ago

#26919 closed enhancement (fixed)

Upgrade Jupyter notebook to latest (6.1.1) and its dependencies to latest

Reported by: Samuel Lelièvre Owned by:
Priority: critical Milestone: sage-9.2
Component: packages: standard Keywords: upgrade, jupyter, notebook
Cc: Antonio Rojas, François Bissey, Bryan Gin-ge Chen, Timo Kaufmann, Ximin Luo, Jeroen Demeyer, Samuel Lelièvre, Tobias Hansen, Eric Gourgoulhon Merged in:
Authors: Matthias Koeppe, Joshua Campbell Reviewers: Joshua Campbell, Matthias Koeppe, Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 445c1d7 (Commits, GitHub, GitLab) Commit: 445c1d7a5f90ddb6af5b33e1d03697cfc3eefc54
Dependencies: #28197, #30299, #25363 Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

Upgrade:

New dependencies:

  • argon2-cffi (?)
  • pyrsistent, attrs, importlib_metadata, zipp

Upgrade of its dependencies:

Recursive dependencies:

Upgrade of a reverse dependency:

(tarball URLs: see checksums.ini; use ./configure --enable-download-from-upstream-url to enable automatic downloads of the new versions)

Previous upgrades:

  • notebook 5.7.6 in #27463
  • notebook 5.7.1, in #26709, merged in Sage 8.5.beta5.

See also:

  • #30124 System package information, spkg-configure for Jupyter "notebook" package and dependencies
  • #30306 Meta-ticket: Use system Jupyter

Change History (58)

comment:1 Changed 4 years ago by Erik Bray

Milestone: sage-8.6sage-8.7

Retarging tickets optimistically to the next milestone. If you are responsible for this ticket (either its reporter or owner) and don't believe you are likely to complete this ticket before the next release (8.7) please retarget this ticket's milestone to sage-pending or sage-wishlist.

comment:2 Changed 4 years ago by Erik Bray

Milestone: sage-8.7sage-8.8

Moving all blocker/critical issues from 8.7 to 8.8.

comment:3 Changed 4 years ago by Samuel Lelièvre

Description: modified (diff)
Summary: Upgrade to Jupyter notebook 5.7.4Upgrade to Jupyter notebook 5.7.7

comment:4 Changed 4 years ago by Samuel Lelièvre

Description: modified (diff)
Summary: Upgrade to Jupyter notebook 5.7.7Upgrade to Jupyter notebook 5.7.8

comment:5 Changed 4 years ago by Samuel Lelièvre

Branch: u/slelievre/notebook-5.7.8

comment:6 Changed 4 years ago by Samuel Lelièvre

Commit: b1a4b7fd4a215fb1340956b6a9e0c15d8ea0c760
Status: newneeds_review

New commits:

b1a4b7fUpgrade to notebook 5.7.8

comment:7 Changed 4 years ago by Erik Bray

Note, this would conflict slightly with #27715. As this ticket came first I'd be happy to rebase onto it so long as do intend to move forward on this ticket (I don't see why we shouldn't, though I haven't tested it yet...)

comment:8 Changed 3 years ago by Erik Bray

Milestone: sage-8.8sage-8.9

Moving tickets from the Sage 8.8 milestone that have been actively worked on in the last six months to the next release milestone (optimistically).

comment:9 Changed 3 years ago by Frédéric Chapoton

branch is red

comment:10 Changed 3 years ago by Frédéric Chapoton

Status: needs_reviewneeds_work

comment:11 Changed 3 years ago by Erik Bray

Milestone: sage-8.9sage-9.1

Ticket retargeted after milestone closed

comment:12 Changed 3 years ago by Matthias Köppe

Milestone: sage-9.1sage-9.2

comment:13 Changed 2 years ago by Samuel Lelièvre

Notebook 6.1.0 RC1 is out:

Notebook 5.7.10 would be the last py2-compatible release.

comment:14 Changed 2 years ago by Matthias Köppe

py2 compatibility is irrelevant now. Please see #29141 for discussion what we need to be compatible with

comment:15 Changed 2 years ago by Samuel Lelièvre

comment:16 Changed 2 years ago by Matthias Köppe

Summary: Upgrade to Jupyter notebook 5.7.8Upgrade to Jupyter notebook 5.7.10 or 6.1.0 ....

comment:17 Changed 2 years ago by Matthias Köppe

See also: #28313 "notebook-5.7.6 contains local files"

comment:18 Changed 2 years ago by Matthias Köppe

Cc: Eric Gourgoulhon added
Description: modified (diff)

comment:19 Changed 2 years ago by Matthias Köppe

Dependencies: #28197
Description: modified (diff)
Summary: Upgrade to Jupyter notebook 5.7.10 or 6.1.0 ....Upgrade Jupyter notebook to latest (6.1.1) and its dependencies to latest

comment:20 Changed 2 years ago by Matthias Köppe

Branch: u/slelievre/notebook-5.7.8u/mkoeppe/notebook-5.7.8

comment:21 Changed 2 years ago by Matthias Köppe

Commit: b1a4b7fd4a215fb1340956b6a9e0c15d8ea0c7606ad6e9b81d1e35aa982ab024a5bf8f7b9ed8d2f0
  [notebook-6.1.1]   Applying patches from ../patches...
  [notebook-6.1.1]   Applying ../patches/0001-Workaround-for-wrong-errno-on-socket-bind-permission.patch
  [notebook-6.1.1]   patching file notebook/notebookapp.py
  [notebook-6.1.1]   Hunk #1 FAILED at 1406.

It should be checked if this patch has been upstreamed


Last 10 new commits:

3205576build/pkgs/notebook: Update to 6.1.1
ef9a90dbuild/pkgs/nbformat: Update to 5.0.7
072ea38build/pkgs/nbconvert: Update to 5.6.1
712a5b6build/pkgs/jupyter_client: Update to 6.1.6
00373c6build/pkgs/jinja2: Update to 2.11.2
7e1b5c5build/pkgs/tornado: Update to 6.1.1
7d73f70build/pkgs/terminado: Update to 0.8.3
c4b19e8build/pkgs/prometheus_client: Update to 0.8.0
9075a16build/pkgs/jupyter_core: Update to 4.6.3
6ad6e9bbuild/pkgs/pyzmq: Update to 19.0.2

comment:22 Changed 2 years ago by Matthias Köppe

The patch was added for Cygwin in #27715 (Jupyter notebook can crash during startup if default port cannot be used)

comment:23 Changed 2 years ago by git

Commit: 6ad6e9b81d1e35aa982ab024a5bf8f7b9ed8d2f026691af0ce7bf0f96a12bc676037e2bda4616f97

Branch pushed to git repo; I updated commit sha1. New commits:

26691afbuild/pkgs/notebook: Remove upstreamed patch (PR 4584)

comment:24 Changed 2 years ago by Matthias Köppe

Status: needs_workneeds_review

Builds OK for me on macOS. I have not tested anything else because I do not use the jupyter notebook and cannot distinguish a working and a broken installation.

comment:25 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:26 Changed 2 years ago by Matthias Köppe

Authors: Matthias Koeppe

comment:27 Changed 2 years ago by git

Commit: 26691af0ce7bf0f96a12bc676037e2bda4616f974762704eb5a089237b18948b85e3171208ee23fd

Branch pushed to git repo; I updated commit sha1. New commits:

4762704build/pkgs/widgetsnbextension: Update to 3.5.1

comment:28 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:29 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:30 Changed 2 years ago by git

Commit: 4762704eb5a089237b18948b85e3171208ee23fd0e1c3258bdb73cdd403daabb0c52d4bb5861269c

Branch pushed to git repo; I updated commit sha1. New commits:

b97ebb2build/pkgs/jsonschema: Update to 3.2.0
4b98e39build/pkgs/mistune: Add upstream_url
4c8e3f6build/pkgs/bleach: Update to 3.1.5
0e1c325build/pkgs/testpath: Update to 0.4.4

comment:31 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:32 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:33 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:34 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:35 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:36 Changed 2 years ago by Matthias Köppe

Dependencies: #28197#28197, #30299

comment:37 Changed 2 years ago by Joshua Campbell

Hello! Saw your post on sage-devel and figured I'd take a shot at testing this.

I was able to get this branch to build just fine, but I ran into some missing run-time dependencies for jsonschema, namely: attrs, importlib_metadata, and pyrsistent (which then required zipp).

I've pushed a commit with these packages added. This is the first time I've done any work with the SPKGs, and I'm not sure if I've followed the conventions correctly, so I didn't want to overwrite your branch in this ticket. For example, I wasn't sure how/whether to include these run-time dependencies in the build/pkgs/{jsonschema|pyrsistent}/dependencies files.

With these packages present, the Jupyter server starts up and notebooks can be opened, edited, and ran just fine. Threejs plots are working including the new animation stuff, but JSmol plots are not. Nor does MathJax function correctly. I see 404 status codes for /nbextensions/mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured and /nbextensions/jsmol/JSmol.min.js, which are perhaps related to some of the other outstanding Jupyter tickets.

comment:38 Changed 2 years ago by Matthias Köppe

Thanks a lot! Please do update the branch, it will make it easier

comment:39 Changed 2 years ago by Matthias Köppe

Yes, runtime dependencies should be added in the dependencies file. We do not really distinguish build dependencies and runtime dependencies in the Sage distribution. (There is a notion of order-only dependencies, which are listed after the vertical-bar symbol (|), but it's OK to ignore.)

comment:40 in reply to:  37 ; Changed 2 years ago by Matthias Köppe

Replying to gh-jcamp0x2a:

I see 404 status codes for /nbextensions/mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured and /nbextensions/jsmol/JSmol.min.js, which are perhaps related to some of the other outstanding Jupyter tickets.

OK, I forgot to push the merge with #30299 -- sorry about that

comment:41 Changed 2 years ago by git

Commit: 0e1c3258bdb73cdd403daabb0c52d4bb5861269c0a6487cd2eaac6c5b57792835a9899bbb437eb56

Branch pushed to git repo; I updated commit sha1. New commits:

113625bbuild/pkgs/ipywidgets: Update to 7.5.1
79317b8build/pkgs/sage_conf/src/sage_conf.py.in: Add JSMOL_DIR, MATHJAX_DIR, THREEJS_DIR
0a6487cMerge branch 't/30299/minimal_fix_for_broken_jupyter_notebook' into t/26919/jupyter-notebook-6.1.1

comment:42 Changed 2 years ago by git

Commit: 0a6487cd2eaac6c5b57792835a9899bbb437eb56af85282a8b93938208c8148d371522a5a4202530

Branch pushed to git repo; I updated commit sha1. New commits:

1414916Add additional run-time dependencies of jsonschema
af85282Merge commit '14149164c43758b8f2845630df96b89dbe1f1c79' of git://trac.sagemath.org/sage into t/26919/jupyter-notebook-6.1.1

comment:43 Changed 2 years ago by Matthias Köppe

I've merged your commit. Please test again...

comment:44 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:45 Changed 2 years ago by Matthias Köppe

comment:46 in reply to:  39 Changed 2 years ago by Joshua Campbell

Replying to mkoeppe:

Yes, runtime dependencies should be added in the dependencies file. We do not really distinguish build dependencies and runtime dependencies in the Sage distribution. (There is a notion of order-only dependencies, which are listed after the vertical-bar symbol (|), but it's OK to ignore.)

Ok, I've added dependencies on attrs, importlib_metadata, and pyrsistent to jsonschema and a dependency on zipp to importlib_metadata. (I mispoke earlier that it was required by pyrsistent.) I will update the ticket with the branch.

Also, I want to mention that the latest version of zipp (3.1.0) did not build correctly in the Sage environment, and I didn't do much troubleshooting on it, falling back instead to the specific major and minor version called out by importlib_metadata (0.5, so I used 0.5.2). Just in case that comes up in the future.

comment:47 in reply to:  40 Changed 2 years ago by Joshua Campbell

Replying to mkoeppe:

Replying to gh-jcamp0x2a:

I see 404 status codes for /nbextensions/mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured and /nbextensions/jsmol/JSmol.min.js, which are perhaps related to some of the other outstanding Jupyter tickets.

OK, I forgot to push the merge with #30299 -- sorry about that

I've tested this again following your merge and both JSmol and MathJax are now working correctly.

comment:48 Changed 2 years ago by Joshua Campbell

Branch: u/mkoeppe/notebook-5.7.8u/gh-jcamp0x2a/notebook-5.7.8
Commit: af85282a8b93938208c8148d371522a5a4202530d7d995ddc8cf9a281b4cf22839986330ab95eea2

comment:49 Changed 2 years ago by Matthias Köppe

Authors: Matthias KoeppeMatthias Koeppe, Joshua Campbell

Thanks very much! Looking great.

I have restarted the build tests, now at https://github.com/mkoeppe/sage/actions/runs/201611495

comment:50 Changed 2 years ago by Matthias Köppe

Reviewers: ​https://github.com/mkoeppe/sage/actions/runs/201611495

comment:51 Changed 2 years ago by Matthias Köppe

Reviewers: ​https://github.com/mkoeppe/sage/actions/runs/201611495Joshua Campbell, ​https://github.com/mkoeppe/sage/actions/runs/201611495

comment:52 Changed 2 years ago by Matthias Köppe

Reviewers: Joshua Campbell, ​https://github.com/mkoeppe/sage/actions/runs/201611495Joshua Campbell, ​github.com/mkoeppe/sage/actions/runs/201611495, ...

Clean tests for slackware-14.2-standard (https://github.com/mkoeppe/sage/runs/964353866), gentoo-standard (https://github.com/mkoeppe/sage/runs/964353835), fedora-32-standard (https://github.com/mkoeppe/sage/runs/964353804). (Tests on many other platforms did not go through because of current reliability issues of GH Action - unrelated to this ticket.)

comment:53 Changed 2 years ago by Matthias Köppe

Reviewers: Joshua Campbell, ​github.com/mkoeppe/sage/actions/runs/201611495, ...Joshua Campbell, Matthias Koeppe, ​github.com/mkoeppe/sage/actions/runs/201611495, ...

comment:54 Changed 2 years ago by Matthias Köppe

Branch: u/gh-jcamp0x2a/notebook-5.7.8u/mkoeppe/notebook-5.7.8

comment:55 Changed 2 years ago by Matthias Köppe

Commit: d7d995ddc8cf9a281b4cf22839986330ab95eea2445c1d7a5f90ddb6af5b33e1d03697cfc3eefc54
Dependencies: #28197, #30299#28197, #30299, #25363

Merged #25363 (and 9.2.beta8) to facilitate testing


New commits:

3b8b2bdtrac 25363: implement "sage --simple-prompt"
445c1d7Merge branch 'u/jhpalmieri/ticket-25363' of git://trac.sagemath.org/sage into t/26919/jupyter-notebook-6.1.1

comment:56 Changed 2 years ago by Dima Pasechnik

Reviewers: Joshua Campbell, Matthias Koeppe, ​github.com/mkoeppe/sage/actions/runs/201611495, ...Joshua Campbell, Matthias Koeppe, ​github.com/mkoeppe/sage/actions/runs/201611495, Dima Pasechnik
Status: needs_reviewpositive_review

Great!

comment:57 Changed 2 years ago by Matthias Köppe

Reviewers: Joshua Campbell, Matthias Koeppe, ​github.com/mkoeppe/sage/actions/runs/201611495, Dima PasechnikJoshua Campbell, Matthias Koeppe, Dima Pasechnik

Thank you!

comment:58 Changed 2 years ago by Volker Braun

Branch: u/mkoeppe/notebook-5.7.8445c1d7a5f90ddb6af5b33e1d03697cfc3eefc54
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.