Opened 9 years ago
Last modified 9 years ago
#13121 closed enhancement
Upgrade sagenb to 0.10.0 — at Version 39
Reported by:  kini  Owned by:  jason, mpatel, was 

Priority:  major  Milestone:  sage5.4 
Component:  notebook  Keywords:  sagenb 
Cc:  jhpalmieri, gutow, jyeo, rbeezer  Merged in:  
Authors:  Keshav Kini  Reviewers:  
Report Upstream:  N/A  Work issues:  
Branch:  Commit:  
Dependencies:  #11080, #9774, #8473, #11913, #12299  Stopgaps: 
Description (last modified by )
This is a followup to #11080. During the review process of #11080, which stretched for quite a while, at some point we decided to have a feature freeze on the version of sagenb 0.9.x associated with #11080, just to simplify the review process. The purpose of this ticket is to incorporate some new features, bugfixes, code refactoring, etc. into the version of sagenb shipped with Sage, beyond what was reviewed in #11080. sagenb 0.10.0 contains the following (among other things):
 refactoring of the startup code, uWSGI and Tornado added as possible WSGI servers
 code allowing .sws files to be uploaded by supplying a URL pointing to the .sws as an argument to the upload page's URL
 new dependency on webassets for js/css autominification (not currently used in the master branch)
 the mathjax branch, and a doctest fix in sphinxify.py (#9774 depends on and contributed to this)
 support for uploading .sws files from the local disk via the command line (#8473 basically is this)
 refactoring of sageinspect to not have a duplication of code between the notebook and the Sage library (#11913 depends on this)
 Upgrading Jmol again and allowing for advanced interactive features more easily, solve some memory issues (#12299)
Changes not in sagenb itself but in how sagenb is incorporated into sage:
 No repository will be installed with sagenb  developers will need to clone sagenb themselves and install the new clone into their Sage installation manually (which is not difficult).
SPKG: http://wstein.org/home/keshav/files/sagenb0.10.0.spkg (a comparatively svelte 13 MB)
Instructions after following the ones at #11080:
cd $SAGE_ROOT ./sage hg R local/bin qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/9774/trac_9774scriptsmathjax.patch ./sage hg R local/bin qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/13121/trac_13121scripts.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/9774/trac_9774mathjaxtry6.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11913/trac_11913sage.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/12299/trac_12299allinone.patch ./sage i http://wstein.org/home/keshav/files/sagenb0.10.0.spkg ./sage i http://www.uwosh.edu/faculty_staff/gutow/jmol12.3.27.p0.spkg ./sage b
Or, instructions without #11080:
cd $SAGE_ROOT ./sage hg R . qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11874/trac11874removetwisted.patch ./sage hg R . qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11503/trac11503jmolspkg.2.patch ./sage hg R local/bin qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/9774/trac_9774scriptsmathjax.patch ./sage hg R local/bin qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11080/trac_11080scriptshgignoredulwich.patch ./sage hg R local/bin qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/13121/trac_13121scripts.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11874/sagespkg11874.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11078/trac_11078.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11503/trac11503jmolcommandline.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/12229/trac_12229sagenbdeveloperdoc.3.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/12229/trac12229manifest.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11080/trac11080notebookdocs.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11080/trac_11080opensslininstallationdocs.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/9774/trac_9774mathjaxtry6.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/11913/trac_11913sage.patch ./sage hg R devel/sage qimport P http://trac.sagemath.org/sage_trac/rawattachment/ticket/12299/trac_12299allinone.patch ./sage i openssl # optional if you have systemwide SSL dev headers installed ./sage i http://wstein.org/home/keshav/files/sagenb0.10.0.spkg ./sage i http://www.uwosh.edu/faculty_staff/gutow/jmol12.3.27.p0.spkg ./sage b
Change History (40)
comment:1 Changed 9 years ago by
 Description modified (diff)
 Milestone changed from sage5.1 to sage5.2
comment:2 Changed 9 years ago by
 Cc jhpalmieri added
comment:3 Changed 9 years ago by
 Description modified (diff)
comment:4 Changed 9 years ago by
 Dependencies changed from #11080 to #11080, #9774, #8473, #11913
 Description modified (diff)
comment:5 Changed 9 years ago by
 Dependencies changed from #11080, #9774, #8473, #11913 to #11080, #9774, #8473, #11913, #12299
 Description modified (diff)
comment:6 Changed 9 years ago by
 Description modified (diff)
 Status changed from new to needs_review
I've released 0.9.1. The SPKG is here. Please test and review!
comment:7 Changed 9 years ago by
 Status changed from needs_review to needs_work
In spkginstall, the check
if [ n $SAGE64 ]; then
needs to take into account that $SAGE64
may be set to "no". According to SAGE_ROOT/spkg/bin/sagearchenv
, it should be unset, "no", or "yes", so you should test that it's set to "yes".
comment:8 Changed 9 years ago by
You should also take this possible change to CPPFLAGS into account. For example:

spkginstall
diff git a/spkginstall b/spkginstall
a b die () { 17 17 export CPATH="$CPATH""$SAGE_LOCAL"/include 18 18 export LIBRARY_PATH="$LIBRARY_PATH""$SAGE_LOCAL"/lib 19 19 20 export CPPFLAGS="I$SAGE_LOCAL/include $CPPFLAGS "20 export CPPFLAGS="I$SAGE_LOCAL/include $CPPFLAGS D_XOPEN_SOURCE=500" 21 21 export LDFLAGS="L$SAGE_LOCAL/lib $LDFLAGS" 22 22 23 if [ n $SAGE64]; then23 if [ $SAGE64 = "yes" ]; then 24 24 echo "Building with extra 64bit flags for MacOS X and Open Solaris." 25 25 if [ z $CFLAG64 ]; then 26 26 CFLAG64=m64
(even better, with a comment explaining why CPPFLAGS has been changed)
comment:9 Changed 9 years ago by
With these changes, this spkg builds on OS X Lion, sage.math, hawk, and skynet machines mark (after installing the openssl spkg from #13126), cicero, iras (with openssl), silius (with openssl), and taurus. So it's looking good to me.
comment:10 Changed 9 years ago by
 Description modified (diff)
comment:11 Changed 9 years ago by
 Status changed from needs_work to needs_review
SPKG updated with your changes, more or less:

spkginstall
diff git a/spkginstall b/spkginstall
a b 17 17 export CPATH="$CPATH""$SAGE_LOCAL"/include 18 18 export LIBRARY_PATH="$LIBRARY_PATH""$SAGE_LOCAL"/lib 19 19 20 export CPPFLAGS="I$SAGE_LOCAL/include $CPPFLAGS" 20 # note: the D_XOPEN_SOURCE=500 is to fix build errors in Twisted 12.1 on 21 # Solaris. See http://trac.sagemath.org/sage_trac/ticket/11080#comment:314 and 22 # the subsequent few comments. 23 export CPPFLAGS="I$SAGE_LOCAL/include D_XOPEN_SOURCE=500 $CPPFLAGS" 21 24 export LDFLAGS="L$SAGE_LOCAL/lib $LDFLAGS" 22 25 23 if [ n $SAGE64]; then26 if [ $SAGE64 = "yes" ]; then 24 27 echo "Building with extra 64bit flags for MacOS X and Open Solaris." 25 28 if [ z $CFLAG64 ]; then 26 29 CFLAG64=m64
comment:12 Changed 9 years ago by
On my Mac (OS X 10.7), I can't get worksheets to actually evaluate anything. With Chrome, Firefox, and Safari, I see an evaluate button, but clicking on it does nothing. Hitting 'shiftenter' either does nothing (Firefox) or gives me a new line in the current cell (Chrome and Safari), and does not evaluate anything. Hitting 'Save & quit' does nothing (Chrome and Safari) or actually quits the current worksheet (Firefox).
I think I see some of the same behavior with 0.9.0, but not as severe: it works in Firefox, but not in Chrome or Safari. I'm not sure about that, though; I'm not sure my 0.9.0 install was clean, or whether I corrupted it with 0.9.1 related things.
Does it matter that there is no directory for this version of sagenb in SAGE_LOCAL/lib/python/sitepackages/
? After installing 0.9.0 and then 0.9.1, I see
drwxrxrx 4 palmieri admin 136 Jun 19 11:13 sagenb0.8.29py2.7.egg drwxrxrx 4 palmieri admin 136 Jun 21 16:47 sagenb0.9.0py2.7.egg rwrr 1 palmieri admin 26 Jun 21 16:59 sagenb.egglink
comment:13 Changed 9 years ago by
comment:14 Changed 9 years ago by
A hard refresh helps to get 0.9.0 to work, but not 0.9.1.
By the way, when the notebook is running, how can I tell which version is running? I would suggest having a variable, or displaying it somewhere when the notebook opens.
comment:15 Changed 9 years ago by
 Description modified (diff)
 Status changed from needs_review to needs_work
 Summary changed from Upgrade sagenb to 0.9.1 to Upgrade sagenb to 0.10
 Work issues set to fix js typo, make new spkg
Um, wow. Silly typo in this commit. I'll fix this, and actually I think I'm going to call this 0.10. Our version numbers are moving too slowly. Adding mathjax is a lot more substantial of a change than a bump to the patch level should cause a user to expect.
comment:16 Changed 9 years ago by
 Summary changed from Upgrade sagenb to 0.10 to Upgrade sagenb to 0.10.0
comment:17 Changed 9 years ago by
 Description modified (diff)
 Report Upstream changed from Fixed upstream, but not in a stable release. to N/A
 Status changed from needs_work to needs_review
 Work issues fix js typo, make new spkg deleted
OK, new SPKG is up.
By the way, when the notebook is running, how can I tell which version is running? I would suggest having a variable, or displaying it somewhere when the notebook opens.
sagenb.misc.misc.SAGENB_VERSION
should tell you that, but it's sort of a hack. I probably could have changed that in the packaging rewrite...
comment:18 Changed 9 years ago by
Okay, now a hard refresh is getting everything to work, on all three browsers. Now, is there any way to set it up to avoid the hard refresh? Otherwise users may be frustrated when they first switch to the new notebook.
comment:19 Changed 9 years ago by
 Description modified (diff)
I've attached a patch to put "webassets" into the .hgignore file.
comment:20 Changed 9 years ago by
Thanks for the .hgignore patch.
As for avoiding hard refreshes, it might be possible to make Flask set some HTTP headers to the date/time when the server was started, to advise clientside caching. I'm not sure of the details, but it's probably not a oneliner like the typo fix I just made was; I'd also consider it more of a feature than a bugfix, so probably it won't go into 0.10.
comment:21 Changed 9 years ago by
comment:22 Changed 9 years ago by
(I've set myself as the author since this ticket is just about making an SPKG  there are many more important contributors to the new version of sagenb than me, for the record.)
comment:23 followup: ↓ 28 Changed 9 years ago by
This is mostly working for me on OS X Lion, except issues with #8473. Otherwise, the spkg looks good to me. People should test it on other platforms.
comment:24 Changed 9 years ago by
 Priority changed from major to blocker
Any objections to making this a blocker for sage5.2? It feels to me like sage0.9.0 is a kind of "beta version" for sage0.10.0, so I would really prefer to have sage0.10.0 in the final release.
comment:25 Changed 9 years ago by
Actually, I think of 0.9.0 as sort of the stable, LTS release where we had a *long* feature freeze and concentrated on stability. 0.10.0 is like the release just after LTS stable release, where we merged in a bunch of new functionality that was put off during the long feature freeze. There are some important bugfixes in 0.10.0 (IIRC), but there is a lot of new functionality in 0.10 too.
comment:26 followup: ↓ 27 Changed 9 years ago by
That said, I would prefer 0.10 to be in 5.2, but I think it's probably more important to have 0.9 out by the time school starts.
There is still one or two rather frustrating known bugs in 0.9 (and 0.10): some sort of leak where worker processes don't get killed, and sometimes creating a new worksheet actually just gets an old worksheet. If we solve those in 0.10.1 or something, it would be great to have that as our beforesemester release.
comment:27 in reply to: ↑ 26 Changed 9 years ago by
That said, I would prefer 0.10 to be in 5.2, but I think it's probably more important to have 0.9 out by the time school starts.
Correct.
comment:28 in reply to: ↑ 23 ; followup: ↓ 29 Changed 9 years ago by
Replying to jhpalmieri:
This is mostly working for me on OS X Lion, except issues with #8473. Otherwise, the spkg looks good to me. People should test it on other platforms.
Just for reference, this was (or will be, when it's merged) dealt with  see this pull request on sagenb.
comment:29 in reply to: ↑ 28 Changed 9 years ago by
Replying to kcrisman:
Replying to jhpalmieri:
This is mostly working for me on OS X Lion, except issues with #8473. Otherwise, the spkg looks good to me. People should test it on other platforms.
Just for reference, this was (or will be, when it's merged) dealt with  see this pull request on sagenb.
Right. Keshav, when you have a chance, could you make a new 0.10.0 spkg for us to test? Any notebook fans: can you positively review the last change at #11080 (the one which allows building on Solaris)?
comment:30 Changed 9 years ago by
To clarify for whoever makes the "final" spkg here  to make sure #8473 is aok, we would need this pull request and this one to fix the minor issues John mentioned.
comment:31 Changed 9 years ago by
I'm confused about the location of JMOL. It used to be (before #11080) part of sagenb. Now it isn't (or at least I don't see it anywhere). Does it need to be distributed as a separate spkg, and thus included in spkg/standard/deps? Or should the new JMOL spkg from #12299 be part of this spkg?
Oh, I see: it is now in spkg/standard/deps, and it needs to be included as a separate spkg.
comment:32 followup: ↓ 34 Changed 9 years ago by
comment:33 Changed 9 years ago by
The spkg here needs work: when it gets installed, SAGE_ROOT/devel
may not yet exist. So something like this:

spkginstall
diff git a/spkginstall b/spkginstall
a b BASENAME=${BASENAME%.tar.gz} 53 53 #easy_install H None f src "sagenb"  die "Error installing sagenb !" 54 54 55 55 # Extract sagenb into $SAGE_ROOT/devel and set up the symlink 56 mkdir p "$SAGE_ROOT/devel" 56 57 cd "$SAGE_ROOT/devel" 57 58 if [ d "$SAGE_ROOT/devel/sagenbmain" ]; then 58 59 echo "Moving old SageNB package to '$SAGE_ROOT/devel/sagenbmainold'..."
comment:34 in reply to: ↑ 32 Changed 9 years ago by
See #11503, where the deps file is patched to require the new JMOL spkg from #11503. For some reason #12299 seems to require a newer version of the SPKG, a version which is not on #11503 or anywhere else. I posted a comment on #11503 about it.
As far as I know, this is on purpose, because the newest Jmol at #12299 needs the many updates to Jmol there. In that sense #11503 is a 'hard' prereq for #11080 and hence this ticket. I am pretty sure Jonathan was careful about exactly which version of Jmol needed to be with which other patches.
comment:35 Changed 9 years ago by
OK, I see, thanks.
comment:36 Changed 9 years ago by
jhpalmieri or others:
When you ran into the problem with the evaluate button not working, did you find that other buttons also stopped working? For instance, the Stop and Archive buttons on the main worksheet list page. I must have done something wrong because even a hard refresh isn't getting things to work for me, but I wanted to ask if anyone had tried those.
comment:37 followup: ↓ 38 Changed 9 years ago by
For me, on some browsers, everything stopped working. A hard refresh did fix it (Firefox or Chrome: cmdshiftr. Safari: hold the option key down while hitting the little "refresh" symbol at the right end of the address box).
comment:38 in reply to: ↑ 37 Changed 9 years ago by
For me, on some browsers, everything stopped working.
Ok.
A hard refresh did fix it (Firefox or Chrome: cmdshiftr. Safari: hold the option key down while hitting the little "refresh" symbol at the right end of the address box).
Yup, that's what I tried, and that didn't work. But like I said on those commits in sagenb, maybe I just destroyed my installation anyway.
Edit: I did finally manage to get the upstream commit of the parenthesis fix in to make it work with the "hard refresh". That definitely needs some big warning somewhere... Strange that doing it "by hand" didn't work, but getting from upstream did.
comment:39 Changed 9 years ago by
 Description modified (diff)
See this comment on github pull request 64 to confirm that #12299 will be in 0.9.1.