Opened 9 years ago
Closed 8 years ago
#11503 closed task (fixed)
Make new spkg to install Jmol in SAGE_LOCAL/share
Reported by:  gutow  Owned by:  gutow 

Priority:  major  Milestone:  sage5.2 
Component:  notebook  Keywords:  sd31 Jmol flask 
Cc:  rado, fbissey, strogdon  Merged in:  sage5.2.beta0 
Authors:  Jonathan Gutow  Reviewers:  Dan Drake, Jason Grout, Punarbasu Purkayastha 
Report Upstream:  N/A  Work issues:  
Branch:  Commit:  
Dependencies:  #11080, #11078, #11874  Stopgaps: 
Description (last modified by )
In an effort to remove Jmol from the notebook in the Flask notebook it is being moved to SAGE_LOCAL/share. This is based on the logic that the notebook can run without Jmol, but Sage cannot run without either the notebook or Jmol even when using just the command line.
To achieve this relocation of Jmol:
 install the spkg http://www.uwosh.edu/faculty_staff/gutow/jmol12.2.21.p0.spkg
 Apply trac11503jmolspkg.2.patch to the sage root repository.
 Apply trac11503jmolcommandline.patch to the sage repository (fixes jmol on the command line)
The necessary patch for sagenb has already been merged with the master sagenb branch: https://github.com/sagemath/sagenb/pull/28
Attachments (7)
Change History (73)
comment:1 Changed 9 years ago by
 Owner changed from jason, mpatel, was to gutow
Changed 9 years ago by
comment:2 Changed 9 years ago by
 Description modified (diff)
comment:3 Changed 9 years ago by
 Description modified (diff)
comment:4 Changed 9 years ago by
 Cc fbissey added
comment:5 Changed 9 years ago by
 Description modified (diff)
comment:6 Changed 9 years ago by
 Description modified (diff)
spkg refactored to better meet Sage standards and remove dead code from spkginstall.
comment:7 Changed 9 years ago by
 Description modified (diff)
 Status changed from new to needs_review
comment:8 followup: ↓ 9 Changed 9 years ago by
 Status changed from needs_review to needs_work
Several comments:
 There are some backup files present: SPKG.txt~ and spkginstall~.
 As I noted on #11496, you should copy the jmol shell script to SAGE_LOCAL/bin as part of spkginstall.
 It would be helpful if you could prepare the mercurial repository with the version of hg installed with Sage. When I use the version included with Sage (which is the only version on my computer), I get
$ hg status abort: requirement 'dotencode' not supported!
so I can't check the state of the repo without installing a new version of hg.
comment:9 in reply to: ↑ 8 Changed 9 years ago by
 Hmm...didn't notice the backup files. I'll look into that...
 I could add the Jmol script to the jmol .spkg. The one that is in there is the generic install not the one for SAGE...It seems reasonable that it should not be tracked by SAGE...I will add the SAGE version as a patch to the .spkg. Once I get this done, I believe #11496 should be closed, as it will no longer apply...let me know who to notify about that.
 Not sure if I can manage to successfully backtrack on the Hg repository format. In order to work with the flasknotebook on my machine, I found I needed a system install of Hg and I got the latest. Is this going to be a hold up? Hg has been a real pain compared to the other version control systems I've used. Can somebody point out how to do the backtrack cleanly and quickly. If it takes me a couple of hours to untangle Hg that's more time than it will take me to fix the rest of the .spkg problems and more time than I have for this project at the moment.
Replying to jhpalmieri:
Several comments:  There are some backup files present: SPKG.txt~ and spkginstall~.  As I noted on #11496, you should copy the jmol shell script to SAGE_LOCAL/bin as part of spkginstall.  It would be helpful if you could prepare the mercurial repository with the version of hg installed with Sage. When I use the version included with Sage (which is the only version on my computer), I get
$ hg status abort: requirement 'dotencode' not supported!
so I can't check the state of the repo without installing a new version of hg.
comment:10 Changed 9 years ago by
 Description modified (diff)
comment:11 Changed 9 years ago by
 Status changed from needs_work to needs_review
.p2 addresses the following: * reverted to mercurial version included in Sage (lost all the history in the process).
 jmol launch script is now included in the patches and copied to SAGE_LOCAL/bin.
 I think I removed all the backup files.
comment:12 followup: ↓ 13 Changed 9 years ago by
 Status changed from needs_review to needs_work
I upgraded my version of mercurial so I can look at the repo, and I hope that the version included with Sage will be upgraded soon, too (see #10594). Oh, and now I see that I can open this repo with the old version. Anyway, there are some uncommitted files:
$ hg status ? patches/Jmol.js.patch ? patches/jmol.patch
(So you should run "hg add" and then commit again.)
The file spkginstall needs some work. In particular, any variable which refers to a path should be quoted, in case there are spaces in it. (Right now, we don't support having spaces in the path where Sage is, but when it's easy to allow, we should.) I think this should do it:

spkginstall
diff git a/spkginstall b/spkginstall
a b if [ $? ne 0 ]; then 27 27 fi 28 28 29 29 # Check for sagenb location 30 cd $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol"30 cd "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol" 31 31 if [ $? ne 0 ]; then 32 32 echo "No old Jmol install in notebook. Skipping removal of Jmol from notebook." 33 33 else 34 34 echo "Removing Jmol files from the notebook data directory..." 35 rm r $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol"35 rm r "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol" 36 36 fi 37 37 38 38 TEMPDIR=$SPKDIR"/src" … … if [ $? ne 0 ]; then 47 47 echo "Directory "$SAGE_LOCAL"/share/jmol does not exist. Creating Directory..." 48 48 else 49 49 echo "Deleting all files from "$SAGE_LOCAL"/share/jmol..." 50 rm r $SAGE_LOCAL"/share/jmol"50 rm r "$SAGE_LOCAL/share/jmol" 51 51 echo "replacing jmol directory and contents..." 52 52 fi 53 53 54 mkdir $SAGE_LOCAL"/share/jmol"54 mkdir "$SAGE_LOCAL/share/jmol" 55 55 56 56 TEMPDIR=$SPKDIR"/src/jmol" 57 57 cd "$TEMPDIR" 58 cp r * $SAGE_LOCAL"/share/jmol/"58 cp r * "$SAGE_LOCAL/share/jmol/" 59 59 60 60 cd $SAGE_LOCAL"/bin" 61 61 if [ $? ne 0 ]; then … … if [ $? ne 0 ]; then 64 64 else 65 65 echo "Copying jmol script to "$SAGE_LOCAL"/bin." 66 66 cd "$TEMPDIR" 67 cp f jmol $SAGE_LOCAL"/bin"67 cp f jmol "$SAGE_LOCAL/bin" 68 68 fi 69 69 70 70 echo "Installing applet web directory" 71 mkdir $SAGE_LOCAL"/share/jmol/appletweb"72 cp Jmol.js $SAGE_LOCAL"/share/jmol/appletweb"71 mkdir "$SAGE_LOCAL/share/jmol/appletweb" 72 cp Jmol.js "$SAGE_LOCAL/share/jmol/appletweb" 73 73 74 74 TEMPDIR=$SPKDIR"/patches/appletweb" 75 75 cd "$TEMPDIR" … … if [ $? ne 0 ]; then 77 77 echo "Error finding patches/appletweb directory. Exiting." 78 78 exit 1 79 79 fi 80 cp r * $SAGE_LOCAL"/share/jmol/appletweb"80 cp r * "$SAGE_LOCAL/share/jmol/appletweb" 81 81 82 82 if [ $? ne 0 ]; then 83 83 echo "Error installing PACKAGE_NAME."
comment:13 in reply to: ↑ 12 Changed 9 years ago by
 Status changed from needs_work to needs_review
The below is done. Try downloading the spkg again after 3 PM CDT.
Replying to jhpalmieri:
I upgraded my version of mercurial so I can look at the repo, and I hope that the version included with Sage will be upgraded soon, too (see #10594). Oh, and now I see that I can open this repo with the old version. Anyway, there are some uncommitted files:
$ hg status ? patches/Jmol.js.patch ? patches/jmol.patch
(So you should run "hg add" and then commit again.) The file spkginstall needs some work. In particular, any variable which refers to a path should be quoted, in case there are spaces in it. (Right now, we don't support having spaces in the path where Sage is, but when it's easy to allow, we should.) I think this should do it:#!diff diff git a/spkginstall b/spkginstall  a/spkginstall +++ b/spkginstall @@ 27,12 +27,12 @@ if [ $? ne 0 ]; then fi # Check for sagenb location cd $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol" +cd "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol" if [ $? ne 0 ]; then echo "No old Jmol install in notebook. Skipping removal of Jmol from notebook." else echo "Removing Jmol files from the notebook data directory..."  rm r $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol" + rm r "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol" fi TEMPDIR=$SPKDIR"/src" @@ 47,15 +47,15 @@ if [ $? ne 0 ]; then echo "Directory "$SAGE_LOCAL"/share/jmol does not exist. Creating Directory..." else echo "Deleting all files from "$SAGE_LOCAL"/share/jmol..."  rm r $SAGE_LOCAL"/share/jmol" + rm r "$SAGE_LOCAL/share/jmol" echo "replacing jmol directory and contents..." fi mkdir $SAGE_LOCAL"/share/jmol" +mkdir "$SAGE_LOCAL/share/jmol" TEMPDIR=$SPKDIR"/src/jmol" cd "$TEMPDIR" cp r * $SAGE_LOCAL"/share/jmol/" +cp r * "$SAGE_LOCAL/share/jmol/" cd $SAGE_LOCAL"/bin" if [ $? ne 0 ]; then @@ 64,12 +64,12 @@ if [ $? ne 0 ]; then else echo "Copying jmol script to "$SAGE_LOCAL"/bin." cd "$TEMPDIR"  cp f jmol $SAGE_LOCAL"/bin" + cp f jmol "$SAGE_LOCAL/bin" fi echo "Installing applet web directory" mkdir $SAGE_LOCAL"/share/jmol/appletweb" cp Jmol.js $SAGE_LOCAL"/share/jmol/appletweb" +mkdir "$SAGE_LOCAL/share/jmol/appletweb" +cp Jmol.js "$SAGE_LOCAL/share/jmol/appletweb" TEMPDIR=$SPKDIR"/patches/appletweb" cd "$TEMPDIR" @@ 77,7 +77,7 @@ if [ $? ne 0 ]; then echo "Error finding patches/appletweb directory. Exiting." exit 1 fi cp r * $SAGE_LOCAL"/share/jmol/appletweb" +cp r * "$SAGE_LOCAL/share/jmol/appletweb" if [ $? ne 0 ]; then echo "Error installing PACKAGE_NAME."
comment:14 Changed 9 years ago by
 Description modified (diff)
 Milestone changed from sage4.8 to sage5.0
Removing command line patch as it appears to be incorporated into 4.8.
comment:15 Changed 9 years ago by
 Status changed from needs_review to needs_work
I'm looking this over, and I have some more minor nits to pick with spkginstall. Here's an idiom I see a couple times:
TEMPDIR=something cd "$TEMPDIR" do some stuff ...
Two things bother me about that: (1) you call it TEMPDIR
, which implies to me that it's a temporary directory used for some scratch work, but you're talking about an existing directory with important files in it. (2) You never use the TEMPDIR
variable, except in the cd
command...so why not just cd
to the directory?
Also, I think it's more readable to just put the paths into commands, rather than cd
ing around. I would prefer to see
cp $SAGE_FOO/dir/file.txt $SAGE_BAR/dir/dir2/
rather than
cd $SAGE_FOO/dir cp file.txt $SAGE_BAR/dir/dir2/
It makes it easier to read the script, since you don't have to remember what the working directory is. Also, you sometimes change directories inside an ifthenelse, so the working directory depends on what happened there!
Finally, at the end of spkginstall
, you have "PACKAGE NAME" instead of Jmol!
comment:16 followup: ↓ 17 Changed 9 years ago by
 Status changed from needs_work to needs_review
I've looked over the spkg, and modulo my fixes to spkginstall
, I can give it a positive review. My changes are in attachment:trac11503.patch and I have a spkg with those changes at http://sagenb.kaist.ac.kr/~drake/Jmol12.0.45.p3.spkg. It's identical to Jonathan's version except for the changes in attachment:trac11503.patch.
Jason, you volunteered to look it over, but that shouldn't stop anyone else. :)
comment:17 in reply to: ↑ 16 Changed 9 years ago by
 Description modified (diff)
Dan's changes look good to me (obviously much more experience writing these scripts than I have:)) I suggest we try Dan's .spkg for the review and have changed the description to reflect this.
Jonathan
Replying to ddrake:
I've looked over the spkg, and modulo my fixes to
spkginstall
, I can give it a positive review. My changes are in attachment:trac11503.patch and I have a spkg with those changes at http://sagenb.kaist.ac.kr/~drake/Jmol12.0.45.p3.spkg. It's identical to Jonathan's version except for the changes in attachment:trac11503.patch. Jason, you volunteered to look it over, but that shouldn't stop anyone else. :)
comment:18 Changed 9 years ago by
Dan: your changes look fine to me. Do you give everything else a positive review?
comment:19 Changed 9 years ago by
 Status changed from needs_review to positive_review
Oh, wait, I see that you just said modulo your fixes, you give the spkg a positive review. So it sounds like we agree: positive review.
comment:20 Changed 9 years ago by
 Reviewers set to Dan Drake, Jason Grout
comment:21 Changed 9 years ago by
 Description modified (diff)
comment:22 Changed 9 years ago by
 Status changed from positive_review to needs_work
hmm...nothing has been added to the deps file. There should be another patch for that.
comment:23 Changed 9 years ago by
 Status changed from needs_work to needs_review
So I think what needs to be done at this point is:
 Make the jmol spkg not start with a capital letter (simple rename of the spkg file)
 Apply the trac11503jmolspkg.patch to the sage root repository, make an sdist, put the jmol spkg in there, and try building. In essence, review the changes to the sage root repository.
comment:24 Changed 9 years ago by
 Description modified (diff)
comment:25 Changed 9 years ago by
 Dependencies changed from flask notebook to flask notebook, #11874
comment:26 Changed 9 years ago by
 Status changed from needs_review to needs_work
 Work issues set to SPKG.txt
For SPKG.txt
, you should follow the format at http://sagemath.org/doc/developer/producing_spkgs.html#creatinganewspkg
comment:27 followup: ↓ 28 Changed 9 years ago by
Okay, it's not in the documentation, but you should format the Changelog with lines like
=== mypackage3.18.spkg (Jeroen Demeyer, 17 January 2012) === * Changed foo to bar
This is how every spkg does it (except for sagenb, which I can live with).
comment:28 in reply to: ↑ 27 Changed 9 years ago by
Should I change this or will you Dan?
A comment. As a relatively newer developer, I do not appreciate formatting requirements that are not documented. I followed the example in the developer's guide.
Jonathan
Replying to jdemeyer:
Okay, it's not in the documentation, but you should format the Changelog with lines like
=== mypackage3.18.spkg (Jeroen Demeyer, 17 January 2012) === * Changed foo to bar
This is how every spkg does it (except for sagenb, which I can live with).
comment:29 Changed 9 years ago by
 Description modified (diff)
 Status changed from needs_work to needs_review
I made an spkg that reformats the changelog and makes the spkg lowercase. See the new instructions.
Jonathan: I understandI didn't object before because it wasn't in the docs.
comment:30 followup: ↓ 33 Changed 9 years ago by
Yeah, I can imagine it's annoying. It should be documented better. I always thought it was documented, for the simple reason that every spkg is doing it the same way.
The docs should probably be clarified.
comment:31 Changed 9 years ago by
 Description modified (diff)
I made some further improvements to SPKG.txt
, in particular I simplified the Dependencies to
* Sage Notebook (sagenb)
comment:32 Changed 9 years ago by
I've updated the instructions over at #11080 to reflect your new spkg.
comment:33 in reply to: ↑ 30 ; followup: ↓ 34 Changed 9 years ago by
Replying to jdemeyer:
Yeah, I can imagine it's annoying. It should be documented better.
See http://trac.sagemath.org/sage_trac/attachment/ticket/9419/trac_9419part2.patch.
comment:34 in reply to: ↑ 33 Changed 9 years ago by
Yeah, I can imagine it's annoying. It should be documented better.
See http://trac.sagemath.org/sage_trac/attachment/ticket/9419/trac_9419part2.patch.
I knew this sounded familiar!
comment:35 Changed 9 years ago by
 Description modified (diff)
Attached patch which fixes commandline jmol. This issue was discovered over on #9238; I used the fix from the comments there.
comment:36 Changed 9 years ago by
To review this ticket, this is what is left:
 My and jdemeyer's changes to the spkg (reformatting SPKG.txt, basically)
 the trac11503jmolcommandline.patch that fixes commandline jmol
comment:37 Changed 9 years ago by
 Work issues SPKG.txt deleted
comment:38 Changed 9 years ago by
Since jmol doesn't use setuptools, it should be listed normally between the dependencies. My new trac11503jmolspkg.patch does this.
comment:39 Changed 9 years ago by
 Keywords changed from sd31,Jmol, flask to sd31 Jmol flask
Jason, what did you base your commandline patch on? It doesn't apply to 4.8.rc0, even with fuzz, since there seem to be some different lines in the context of the second chunk. Here's a rebase on 4.8.rc0, though I don't know what's going on here so I won't put it in the ticket description.
comment:40 Changed 9 years ago by
 Dependencies changed from flask notebook, #11874 to flask notebook, #11078, #11874
I made the patch on 5.0something (prealpha/beta1 maybe?). I also had #11078 applied before this.
comment:41 Changed 9 years ago by
Oh, OK. Then there is no rebasing needed  4.8.rc0 > #11078 > your patch works perfectly.
comment:42 Changed 9 years ago by
 Status changed from needs_review to needs_work
I tried building a clean sage with this spkg and the new notebook, and commandline jmol didn't work because the local/bin/jmol script was not executable. So it should probably be set to executable inside the spkg.
comment:43 Changed 9 years ago by
 Work issues set to jmol script chmod'd to executable
comment:44 Changed 9 years ago by
 Description modified (diff)
 Status changed from needs_work to needs_review
I just uploaded a p4 version that sets the jmol script to executable after copying to local/bin. So it's needs review again.
comment:45 Changed 9 years ago by
 Cc strogdon added
comment:46 Changed 8 years ago by
The attachment trac11503jmolspkg.2.patch has been rebased to sage5.0beta11. Compiling it now (along with #11874 and the patches to SAGE_ROOT
from #11080).
comment:47 Changed 8 years ago by
 Description modified (diff)
comment:48 Changed 8 years ago by
 Milestone changed from sage5.0 to sage5.1
comment:49 Changed 8 years ago by
 Description modified (diff)
Updated to Jmol12.2.21...this will allow using Jmol to generate images serverside.
comment:50 followup: ↓ 52 Changed 8 years ago by
 Status changed from needs_review to needs_work
 Work issues jmol script chmod'd to executable deleted
$ hg status ? SPKG.txt~
comment:51 Changed 8 years ago by
Since it's a new jmol version, the pversion number should be reset.
comment:52 in reply to: ↑ 50 Changed 8 years ago by
Odd that the chmod is gone...I didn't edit the script....All I did was replace the .jar files....I'll check this afternoon. Not sure why the SPKG backup got left behind, but I can delete it....
Replying to jdemeyer:
$ hg status ? SPKG.txt~
comment:53 Changed 8 years ago by
 Dependencies changed from flask notebook, #11078, #11874 to #11080, #11078, #11874
comment:54 Changed 8 years ago by
 Description modified (diff)
 Status changed from needs_work to needs_review
SPKG.txt~ removed and renamed to ...p0
comment:55 Changed 8 years ago by
I believe this ticket is actually included in the positively reviewed #11080. If there are no objections, I believe this should become a positive review as well.
comment:56 Changed 8 years ago by
 Status changed from needs_review to positive_review
Indeed this should be positive review. I have been using it both from the notebook and from the command line without issues, for several months.
Setting it to positive review.
comment:57 Changed 8 years ago by
 Reviewers changed from Dan Drake, Jason Grout to Dan Drake, Jason Grout, Punarbasu Purkayastha
comment:58 Changed 8 years ago by
 Milestone changed from sage5.1 to sage5.2
comment:59 Changed 8 years ago by
Please put the latest version of the SPKG up on this ticket. It looks like #12299's instructions mention a newer version of the SPKG than is found here.
comment:60 Changed 8 years ago by
comment:61 Changed 8 years ago by
OK, fair enough.
comment:62 Changed 8 years ago by
 Merged in set to sage5.2.beta0
 Resolution set to fixed
 Status changed from positive_review to closed
comment:63 Changed 8 years ago by
 Merged in sage5.2.beta0 deleted
 Milestone changed from sage5.2 to sagepending
 Resolution fixed deleted
 Status changed from closed to new
Unmerging this from sage5.2 due to the serious security issue at #13270.
comment:64 Changed 8 years ago by
 Status changed from new to needs_review
comment:65 Changed 8 years ago by
 Status changed from needs_review to positive_review
comment:66 Changed 8 years ago by
 Merged in set to sage5.2.beta0
 Milestone changed from sagepending to sage5.2
 Resolution set to fixed
 Status changed from positive_review to closed
path fix for flask