#22431 closed enhancement (fixed)
upgrade sagenb and build sagenb in sage/python3
Reported by:  chapoton  Owned by:  

Priority:  major  Milestone:  sage8.3 
Component:  python3  Keywords:  
Cc:  dimpase, vbraun, jdemeyer, kcrisman, embray  Merged in:  
Authors:  Dima Pasechnik  Reviewers:  Frédéric Chapoton, Dima Pasechnik 
Report Upstream:  N/A  Work issues:  
Branch:  ef854c1 (Commits)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
sagenb is not compatible with python3, at least in the current version 1.0.1
The latest release 1.0.2 is certainly slightly better in this respect, but maybe still not fully py3 compliant.
Let us do the upgrade to 1.0.2. We also reactivate the build of sagenb in sagePy3, to help building the doc of sagePy3.
Side remark: to cut the ties between sagenb and sage, we need to get rid:
 of the cluster interact #24760 and #25074
 of the mandelbrot interact #24994
 of the graph editor #?????
 of the graph database navigator #?????
 of the interactive debugger #?????
 of calls to some tab completion function #24998
tarball with sagenb update: https://github.com/sagemath/sagenb/releases/download/1.0.2/sagenb1.0.2.tar.bz2
Change History (63)
comment:1 Changed 3 years ago by
 Cc dimpase vbraun jdemeyer added
 Description modified (diff)
comment:2 Changed 3 years ago by
 Cc kcrisman added
comment:3 Changed 3 years ago by
comment:4 Changed 3 years ago by
I did a lot of work on python3 compliance of sagenb since the last release. At least this should prevent the build to stop on syntax errors.
comment:5 Changed 3 years ago by
IMHO one does not need supernatural resurrection powers to port a dead python2 project of sagenb size to python3given that all of its python components are python3 compatible (which was not true for Twisted until a couple of years back).
comment:6 Changed 3 years ago by
The update of sagenb is probably dependent of #20922.
comment:7 Changed 3 years ago by
 Milestone changed from sage7.6 to sage8.0
comment:8 Changed 3 years ago by
can we hope to make a (maybe last) release of the legacy notebook ?
comment:9 Changed 3 years ago by
At the least https://github.com/sagemath/sagenb/pull/416 and your own https://github.com/sagemath/sagenb/pull/423 would need to be merged first, right?
I don't really see any reason to not keep the legacy notebook around for quite some time, granting that it will not receive much attention. We just got a support request from someone using Sage 4.6 (!) who really didn't want to upgrade; this is positively new compared to that.
But I definitely agree that whatever the absolute minimum needed to get sagenb to work minimally with python3 is fine.
comment:10 Changed 3 years ago by
https://github.com/sagemath/sagenb/pull/423 should now be ready
comment:11 followup: ↓ 12 Changed 3 years ago by
Would it be possible to make a new release of sagenb, please ?
comment:12 in reply to: ↑ 11 Changed 3 years ago by
comment:13 Changed 3 years ago by
Please test https://github.com/sagemath/sagenb/tree/1.0.rc0 (copy of the current master) before I go ahead with making a new Sage package. It works for me following the instructions in HACKING (updated, to take care of changed names and of the need to deal with mathjax).
Report issues on github, please
comment:14 followup: ↓ 15 Changed 3 years ago by
the new sagenb is on #23066 and needs review.
comment:15 in reply to: ↑ 14 ; followup: ↓ 16 Changed 3 years ago by
comment:16 in reply to: ↑ 15 ; followup: ↓ 17 Changed 3 years ago by
Replying to kcrisman:
the new sagenb is on #23066 and needs review.
Reading the comments there and at #22787 make it unclear to me whether this one is, in fact, py3 compatible.
Given that Sage does not run with Python 3 yet, it's a bit early to say, no?
Probably not really, for it does work without #20922  I don't get doctest failures mentoined in comment 4 on #20922.
$ sage t long local/lib/python2.7/sitepackages/sagenb/notebook/challenge.py Running doctests with ID 20170525055719d9e71fa3. Git branch: sagenb10 Using optional=ccache,database_gap,gap_packages,mpir,python2,sage Doctesting 1 file. sage t long warnlong 117.7 local/lib/python2.7/sitepackages/sagenb/notebook/challenge.py [113 tests, 0.38 s]  All tests passed!
I also do not see any mention of click
package in sagenb's installation log.
comment:17 in reply to: ↑ 16 Changed 3 years ago by
the new sagenb is on #23066 and needs review.
Reading the comments there and at #22787 make it unclear to me whether this one is, in fact, py3 compatible.
Given that Sage does not run with Python 3 yet, it's a bit early to say, no?
Haha, I thought we were closer to that now. No worries, just checking in.
comment:18 Changed 23 months ago by
 Description modified (diff)
 Milestone changed from sage8.0 to sage8.3
comment:19 Changed 23 months ago by
see also https://github.com/sagemath/sagenb/issues/440
 some of these dependencies should go, indeed.
comment:20 Changed 22 months ago by
We should also get rid of the sagenb documentation building, which does not work in the python3 sage..
See
comment:21 Changed 21 months ago by
see #25382
comment:22 Changed 21 months ago by
What is the current status of this ticket (as opposed to the documentation build)? I thought at least some of these issues were dealt with. If it's just the graph functionality those can certainly be disabled, as I don't think they have worked properly for some time. If it's certain dependencies that is required for py3 that is somewhat different.
comment:23 Changed 21 months ago by
trying to install sagenb, one gets
[sagenb1.0.1] Removing source in /tmp/pippzy8td0fbuild [sagenb1.0.1] Successfully installed sagenb1.0.1 [sagenb1.0.1] Cleaning up... [sagenb1.0.1] ./spkginstall: line 56: cd: /home/chapoton/sage3/local/lib/python2.7/sitepackages/sagenb/data: No such file or directory [sagenb1.0.1] Error: Cannot find SageNB data directory. [sagenb1.0.1]
because it is looking for sageNB data in the python2 library..
comment:24 Changed 21 months ago by
Dima, could you please make a new release of sagenb ?
Currently (apart from the mathjax issue above in spkginstall) sagenb 1.0.1 installs with sage3, but fails to run because of
Traceback (most recent call last): File "/home/chapoton/sage3/src/bin/sagenotebook", line 268, in <module> launcher(unknown) File "/home/chapoton/sage3/src/bin/sagenotebook", line 69, in __init__ from sagenb.notebook.notebook_object import notebook File "/home/chapoton/sage3/local/lib/python3.6/sitepackages/sagenb/notebook/notebook_object.py", line 23, in <module> from . import run_notebook File "/home/chapoton/sage3/local/lib/python3.6/sitepackages/sagenb/notebook/run_notebook.py", line 21, in <module> from exceptions import SystemExit ModuleNotFoundError: No module named 'exceptions'
which has been fixed after the release of 1.0.1
comment:25 Changed 21 months ago by
OK, I'll try today. Sorry for sitting on this...
comment:26 Changed 21 months ago by
see also #25394
comment:27 Changed 21 months ago by
 Description modified (diff)
comment:28 Changed 21 months ago by
shouldn't the description say "current 1.0.1" ? (sorry, I'm still not done with a new release)
comment:29 Changed 21 months ago by
 Description modified (diff)
comment:30 Changed 21 months ago by
 Branch set to u/dimpase/sagenb/1.0.2
 Commit set to 64a1b0c301469e36e8c29266b06fcdcb16b51d5f
 Status changed from new to needs_review
comment:31 Changed 21 months ago by
Thanks for the release, Dima. It will not yet be fully py3compatible, but this should not prevent us from being able to install sagenb in py3sage.
This second part means : first remove the conditional statement on PYTHON3 in spkg_install (on top of the changes in #25394) and do something for the lines about mathjax in this spkginstall. I managed to install sagenb in py3 sage by commenting out these mathjax lines, but this is obviously not a correct solution.
comment:32 followup: ↓ 35 Changed 21 months ago by
I have put a tentative branch at public/ticket/22431 for the changes in spkginstall (it needs work).
comment:33 followup: ↓ 37 Changed 21 months ago by
What would "full py3 compatibility" entail? Just curious. THANK YOU for this work.
comment:34 Changed 21 months ago by
 Commit changed from 64a1b0c301469e36e8c29266b06fcdcb16b51d5f to 48976b514838a6d7ebac2c06c839a1f367b76652
comment:35 in reply to: ↑ 32 Changed 21 months ago by
Replying to chapoton:
I have put a tentative branch at public/ticket/22431 for the changes in spkginstall (it needs work).
OK, I've merged this branch into the branch on the ticket.
comment:36 followup: ↓ 38 Changed 21 months ago by
 Cc embray added
Then there probably just remains to understand what to do with the line
ln s ../../../../../share/mathjax/ mathjax
This way to find the mathjax package seem to be very ugly... Maybe Erik would have an idea for a better replacement that would also work for python3 ?
comment:37 in reply to: ↑ 33 Changed 21 months ago by
Replying to kcrisman:
What would "full py3 compatibility" entail? Just curious. THANK YOU for this work.
Well, this just means that somebody has to install sagenb on py3sage (using the branch here and also the hacking instructions for sagenb to work with a git branch for sagenb) and try running "sage n=sagenb", fixing step by step all errors (in sagenb code) that occurs until the notebook finally starts. This probably implies caring about unicode (str versus bytes). But there could also be other bad surprises as well. I have started to do that, but have very urgent other matters to handle.
comment:38 in reply to: ↑ 36 Changed 21 months ago by
Replying to chapoton:
Then there probably just remains to understand what to do with the line
ln s ../../../../../share/mathjax/ mathjaxThis way to find the mathjax package seem to be very ugly... Maybe Erik would have an idea for a better replacement that would also work for python3 ?
Why don't we use ln s $SAGE_LOCAL/share/mathjax mathjax
? And is that any better?
comment:39 Changed 21 months ago by
Probably that's what should have been done in the first place, but removing MathJax from sagenb proper was done pretty quickly and this "just worked"? That's my guess.
comment:40 Changed 21 months ago by
Also, on OS X at least, the command should be ln s h ...
– as it is, if the mathjax link already exists, then the current command creates another link within that linked folder. It is okay if the sagenb/data
directory contains
mathjax > ../../../../../share/mathjax/
but it doesn't make sense for $SAGE_LOCAL/share/mathjax
to contain this.
(On OS X, the documentation for the h
flag says "If the target_file or target_dir is a symbolic link, do not follow it." There is also a n
flag: "Same as h, for compatibility with other ln implementations.")
comment:41 Changed 21 months ago by
 Branch changed from u/dimpase/sagenb/1.0.2 to public/ticket/22431
 Commit changed from 48976b514838a6d7ebac2c06c839a1f367b76652 to ef854c1f5b263acf2bab8939370f830314e6c3c4
I have modified the ln to mathjax. This seems to work well with python2. Not yet tested with python3.
We need people that test if the new proposed release of sagenb works well enough. Not much has changed since the last release.
New commits:
ef854c1  change the ln command to mathjax in sagenb spkg

comment:42 Changed 21 months ago by
I have checked that with this branch, the install of sagenb is successful with python3 on sage 8.3.b2.
comment:43 Changed 21 months ago by
ping ?
comment:44 Changed 21 months ago by
I can try building the latest beta with python3,what should I pull in, besides the branch on this ticket?
comment:45 Changed 21 months ago by
Nothing else to pull, the develop branch itself build fine with python3.
comment:46 Changed 21 months ago by
and how do I start sagenb so that the underlying python is python 3?
comment:47 Changed 21 months ago by
Well, just sage n=sagenb
. But it will fail to start, because of twisted or some of the latest commited changes, not yet in the release here.
We only need to make sure that sage3 builds and starts with the branch here.
We also need to make sure that for sage2, the upgrade of sagenb does not break something inside the notebook.
comment:48 Changed 21 months ago by
I propose to rename this ticket to "upgrade sagenb and build sagenb in sage/python3"
and keep for another ticket the goal of being able to run sagenb in sage/python3
comment:49 Changed 21 months ago by
I was going to try this today on py2 to test the notebook but had some troubles upgrading Sage, see sagedevel.
comment:50 Changed 21 months ago by
With python3 sage, I get
$ ./sage n sagenb SageMath version 8.3.beta3, Release Date: 20180527 Please wait while the old SageNB Notebook server starts... /home/dima/sagepy3/local/lib/python3.6/sitepackages/sage/misc/inline_fortran.py:8: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp Traceback (most recent call last): File "/home/dima/sagepy3/src/bin/sagenotebook", line 268, in <module> launcher(unknown) File "/home/dima/sagepy3/src/bin/sagenotebook", line 74, in __init__ notebook(*self.args, **self.kwds) File "/home/dima/sagepy3/local/lib/python3.6/sitepackages/sagenb/notebook/notebook_object.py", line 243, in __call__ return self.notebook(*args, **kwds) File "/home/dima/sagepy3/local/lib/python3.6/sitepackages/sagenb/notebook/run_notebook.py", line 535, in notebook_run nb = notebook.load_notebook(directory) File "/home/dima/sagepy3/local/lib/python3.6/sitepackages/sagenb/notebook/notebook.py", line 1839, in load_notebook nb = Notebook(dir) File "/home/dima/sagepy3/local/lib/python3.6/sitepackages/sagenb/notebook/notebook.py", line 151, in __init__ self.__conf = S.load_server_conf() File "/home/dima/sagepy3/local/lib/python3.6/sitepackages/sagenb/storage/filesystem_storage.py", line 265, in load_server_conf return self._basic_to_server_conf(self._load('conf.pickle')) File "/home/dima/sagepy3/local/lib/python3.6/sitepackages/sagenb/storage/filesystem_storage.py", line 187, in _load result = pickle.load(f) TypeError: a byteslike object is required, not 'str'
comment:51 Changed 21 months ago by
Thanks Dima. It seems that you got a hard time doing that..
Failing is as expected. Maybe this specific issue is already fixed by the pull request done after you made the release. But this is not so important : let us keep that for another ticket.
Do you confirm that sage3 builds and starts with the branch here ? I think I did check that also. So it seems to be good to go from this point of view.
So now, there only remains to see if the branch here is not going to break anything in the sage2 sagenb.
comment:52 Changed 21 months ago by
I have just checked again with sage3 release 8.3.b4 that sage3 builds and starts.
comment:53 Changed 21 months ago by
also, for me with beta4. You can give it a positive review, I think.
comment:54 Changed 21 months ago by
 Description modified (diff)
 Reviewers set to Frédéric Chapoton, Dima Pasechnik
 Status changed from needs_review to positive_review
 Summary changed from py3: sagenb is not python3 compatible to upgrade sagenb and build sagenb in sage/python3
ok, let it be. I change the title and description to be more adequate to what is done here.
comment:55 Changed 21 months ago by
 Branch changed from public/ticket/22431 to ef854c1f5b263acf2bab8939370f830314e6c3c4
 Resolution set to fixed
 Status changed from positive_review to closed
comment:56 followup: ↓ 57 Changed 20 months ago by
 Commit ef854c1f5b263acf2bab8939370f830314e6c3c4 deleted
As a followup, in 8.3.beta6 I am having some troubles  see this post for details. I am pretty sure this isn't due to some stale process.
comment:57 in reply to: ↑ 56 ; followup: ↓ 58 Changed 20 months ago by
Replying to kcrisman:
As a followup, in 8.3.beta6 I am having some troubles  see this post for details. I am pretty sure this isn't due to some stale process.
Does sagenb still open for you directly, via sage n sagenb
?
comment:58 in reply to: ↑ 57 Changed 20 months ago by
As a followup, in 8.3.beta6 I am having some troubles  see this post for details. I am pretty sure this isn't due to some stale process.
Does sagenb still open for you directly, via
sage n sagenb
?
Yes, or at least with sage notebook=sagenb
it does. I think there is something with the browser + token not opening.
comment:59 followup: ↓ 60 Changed 20 months ago by
could you try when undoing #25548 ?
comment:60 in reply to: ↑ 59 ; followup: ↓ 61 Changed 20 months ago by
could you try when undoing #25548 ?
You are oracular.
It would be interesting to trace where exactly quit_sage
is even called that it would be necessary ...
comment:61 in reply to: ↑ 60 Changed 20 months ago by
could you try when undoing #25548 ?
You are oracular.
Maybe I wasn't clear; that fixes it. Shouldn't we have a blocker for 8.3 that includes that?
comment:62 Changed 20 months ago by
I can only say that the description of the issue you give it too vague. Please feel free to open a ticket with details on how to reproduce this.
comment:63 Changed 20 months ago by
Ok; see #25667.
Given that sagenb is essentially a dead project, I don't think that it will ever be Python3 compliant.