Upgrade Jupyter notebook to latest (6.1.1) and its dependencies to latest

[slelievre]

Upgrade:

• ​https://pypi.org/project/notebook/ 6.1.1 (current as of 2020-08-07, supports python >= 3.5)

New dependencies:

• argon2-cffi (?)
• pyrsistent, attrs, importlib_metadata, zipp

Upgrade of its dependencies:

• ​https://pypi.org/project/nbformat/ 5.0.7 (released 2020-06-09, current as of 2020-08-07, supports python >= 3.5)
• ​https://pypi.org/project/nbconvert/ 5.6.1 (released 2019-10-24, current as of 2020-08-07, supports python >= 3.5)
• ​https://pypi.org/project/jupyter-client/ 6.1.6 (released 2020-07-13, current as of 2020-08-07, supports python >= 3.5)
• ​https://pypi.org/project/jupyter-core/ 4.6.3 (released 2020-02-18, current as of 2020-08-07, supports python >= 3.5)
• ​https://pypi.org/project/Jinja2/ 2.11.2 (released 2020-04-13, current as of 2020-08-07, supports python >= 3.5)
• ​https://pypi.org/project/tornado/ 6.0.4 (released 2020-03-03, current as of 2020-08-07, supports python >= 3.5)
• ​https://pypi.org/project/terminado/ 0.8.3 (released 2019-11-12, current as of 2020-08-07, supports python >= 3.4)
• ​https://pypi.org/project/prometheus-client/ 0.8.0 (released 2020-05-25, current as of 2020-08-07, supports python >= 3.4)
• ​https://pypi.org/project/pyzmq/ 19.0.2 (released 2020-07-31, current as of 2020-08-07, supports python >= 3.3)

Recursive dependencies:

• ​https://pypi.org/project/jsonschema/ 3.2.0 (released 2019-11-18, current as of 2020-08-07, supports python >= 2.7)
• bleach 3.1.5
• testpath 0.4.4

Upgrade of a reverse dependency:

• ​https://pypi.org/project/widgetsnbextension/ 3.5.1 (released 2019-07-29, supports python >= 2.7)
• ipywidgets 7.5.1

(tarball URLs: see checksums.ini; use ./configure --enable-download-from-upstream-url to enable automatic downloads of the new versions)

Previous upgrades:

• notebook 5.7.6 in #27463
• notebook 5.7.1, in #26709, merged in Sage 8.5.beta5.

See also:

• #30124 System package information, spkg-configure for Jupyter "notebook" package and dependencies
• #30306 Meta-ticket: Use system Jupyter

[embray]

Retarging tickets optimistically to the next milestone. If you are responsible for this ticket (either its reporter or owner) and don't believe you are likely to complete this ticket before the next release (8.7) please retarget this ticket's milestone to sage-pending or sage-wishlist.

[embray]

Moving all blocker/critical issues from 8.7 to 8.8.

[slelievre]

New commits:

​b1a4b7f

Upgrade to notebook 5.7.8

[embray]

Note, this would conflict slightly with #27715. As this ticket came first I'd be happy to rebase onto it so long as do intend to move forward on this ticket (I don't see why we shouldn't, though I haven't tested it yet...)

[embray]

Moving tickets from the Sage 8.8 milestone that have been actively worked on in the last six months to the next release milestone (optimistically).

[chapoton]

branch is red

[embray]

Ticket retargeted after milestone closed

[slelievre]

Notebook 6.1.0 RC1 is out:

• ​https://discourse.jupyter.org/t/jupyter-notebook-6-1-0-rc1-is-available/4763

Notebook 5.7.10 would be the last py2-compatible release.

• ​https://pypi.org/project/notebook/5.7.10/

[mkoeppe]

py2 compatibility is irrelevant now. Please see #29141 for discussion what we need to be compatible with

[slelievre]

Notebook 6.1.0 released.

• ​Release notes for Jupyter Notebook 6.1.0

[mkoeppe]

See also: #28313 "notebook-5.7.6 contains local files"

[mkoeppe]

  [notebook-6.1.1]   Applying patches from ../patches...
  [notebook-6.1.1]   Applying ../patches/0001-Workaround-for-wrong-errno-on-socket-bind-permission.patch
  [notebook-6.1.1]   patching file notebook/notebookapp.py
  [notebook-6.1.1]   Hunk #1 FAILED at 1406.

It should be checked if this patch has been upstreamed

---

Last 10 new commits:

​3205576	build/pkgs/notebook: Update to 6.1.1
​ef9a90d	build/pkgs/nbformat: Update to 5.0.7
​072ea38	build/pkgs/nbconvert: Update to 5.6.1
​712a5b6	build/pkgs/jupyter_client: Update to 6.1.6
​00373c6	build/pkgs/jinja2: Update to 2.11.2
​7e1b5c5	build/pkgs/tornado: Update to 6.1.1
​7d73f70	build/pkgs/terminado: Update to 0.8.3
​c4b19e8	build/pkgs/prometheus_client: Update to 0.8.0
​9075a16	build/pkgs/jupyter_core: Update to 4.6.3
​6ad6e9b	build/pkgs/pyzmq: Update to 19.0.2

[mkoeppe]

The patch was added for Cygwin in #27715 (Jupyter notebook can crash during startup if default port cannot be used)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​26691af

build/pkgs/notebook: Remove upstreamed patch (PR 4584)

[mkoeppe]

Builds OK for me on macOS. I have not tested anything else because I do not use the jupyter notebook and cannot distinguish a working and a broken installation.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​4762704

build/pkgs/widgetsnbextension: Update to 3.5.1

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​b97ebb2	build/pkgs/jsonschema: Update to 3.2.0
​4b98e39	build/pkgs/mistune: Add upstream_url
​4c8e3f6	build/pkgs/bleach: Update to 3.1.5
​0e1c325	build/pkgs/testpath: Update to 0.4.4

[gh-jcamp0x2a]

Hello! Saw your post on sage-devel and figured I'd take a shot at testing this.

I was able to get this branch to build just fine, but I ran into some missing run-time dependencies for jsonschema, namely: attrs, importlib_metadata, and pyrsistent (which then required zipp).

I've pushed a ​commit with these packages added. This is the first time I've done any work with the SPKGs, and I'm not sure if I've followed the conventions correctly, so I didn't want to overwrite your branch in this ticket. For example, I wasn't sure how/whether to include these run-time dependencies in the build/pkgs/{jsonschema|pyrsistent}/dependencies files.

With these packages present, the Jupyter server starts up and notebooks can be opened, edited, and ran just fine. Threejs plots are working including the new animation stuff, but JSmol plots are not. Nor does MathJax function correctly. I see 404 status codes for /nbextensions/mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured and /nbextensions/jsmol/JSmol.min.js, which are perhaps related to some of the other outstanding Jupyter tickets.

[mkoeppe]

Thanks a lot! Please do update the branch, it will make it easier

[mkoeppe]

Yes, runtime dependencies should be added in the dependencies file. We do not really distinguish build dependencies and runtime dependencies in the Sage distribution. (There is a notion of order-only dependencies, which are listed after the vertical-bar symbol (|), but it's OK to ignore.)

[mkoeppe]

Replying to gh-jcamp0x2a:

I see 404 status codes for /nbextensions/mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured and /nbextensions/jsmol/JSmol.min.js, which are perhaps related to some of the other outstanding Jupyter tickets.

OK, I forgot to push the merge with #30299 -- sorry about that

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​113625b	build/pkgs/ipywidgets: Update to 7.5.1
​79317b8	build/pkgs/sage_conf/src/sage_conf.py.in: Add JSMOL_DIR, MATHJAX_DIR, THREEJS_DIR
​0a6487c	Merge branch 't/30299/minimal_fix_for_broken_jupyter_notebook' into t/26919/jupyter-notebook-6.1.1

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1414916	Add additional run-time dependencies of jsonschema
​af85282	Merge commit '14149164c43758b8f2845630df96b89dbe1f1c79' of git://trac.sagemath.org/sage into t/26919/jupyter-notebook-6.1.1

[mkoeppe]

I've merged your commit. Please test again...

[mkoeppe]

A build test is running at ​https://github.com/mkoeppe/sage/actions/runs/201573039

[gh-jcamp0x2a]

Replying to mkoeppe:

Yes, runtime dependencies should be added in the dependencies file. We do not really distinguish build dependencies and runtime dependencies in the Sage distribution. (There is a notion of order-only dependencies, which are listed after the vertical-bar symbol (|), but it's OK to ignore.)

Ok, I've added dependencies on attrs, importlib_metadata, and pyrsistent to jsonschema and a dependency on zipp to importlib_metadata. (I mispoke earlier that it was required by pyrsistent.) I will update the ticket with the branch.

Also, I want to mention that the latest version of zipp (3.1.0) did not build correctly in the Sage environment, and I didn't do much troubleshooting on it, falling back instead to the specific major and minor version called out by importlib_metadata (0.5, so I used 0.5.2). Just in case that comes up in the future.

[gh-jcamp0x2a]

Replying to mkoeppe:

Replying to gh-jcamp0x2a:

I see 404 status codes for /nbextensions/mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML-full,Safe&delayStartupUntil=configured and /nbextensions/jsmol/JSmol.min.js, which are perhaps related to some of the other outstanding Jupyter tickets.

OK, I forgot to push the merge with #30299 -- sorry about that

I've tested this again following your merge and both JSmol and MathJax are now working correctly.

[mkoeppe]

Thanks very much! Looking great.

I have restarted the build tests, now at ​https://github.com/mkoeppe/sage/actions/runs/201611495

[mkoeppe]

Clean tests for slackware-14.2-standard (​https://github.com/mkoeppe/sage/runs/964353866), gentoo-standard (​https://github.com/mkoeppe/sage/runs/964353835), fedora-32-standard (​https://github.com/mkoeppe/sage/runs/964353804). (Tests on many other platforms did not go through because of current reliability issues of GH Action - unrelated to this ticket.)

[mkoeppe]

Merged #25363 (and 9.2.beta8) to facilitate testing

---

New commits:

​3b8b2bd	trac 25363: implement "sage --simple-prompt"
​445c1d7	Merge branch 'u/jhpalmieri/ticket-25363' of git://trac.sagemath.org/sage into t/26919/jupyter-notebook-6.1.1

[dimpase]

Great!

[mkoeppe]

Thank you!