Make new spkg to install Jmol in SAGE_LOCAL/share

[gutow]

In an effort to remove Jmol from the notebook in the Flask notebook it is being moved to SAGE_LOCAL/share. This is based on the logic that the notebook can run without Jmol, but Sage cannot run without either the notebook or Jmol even when using just the command line.

To achieve this relocation of Jmol:

1. install the spkg  http://www.uwosh.edu/faculty_staff/gutow/jmol-12.2.21.p0.spkg

2. Apply trac-11503-jmol-spkg.2.patch to the sage root repository.

3. Apply trac-11503-jmol-commandline.patch to the sage repository (fixes jmol on the command line)

The necessary patch for sagenb has already been merged with the master sagenb branch:  https://github.com/sagemath/sagenb/pull/28

[gutow]

path fix for flask

[gutow]

plot3d command line fix

[gutow]

spkg refactored to better meet Sage standards and remove dead code from spkg-install.

[jhpalmieri]

Several comments:

• There are some backup files present: SPKG.txt~ and spkg-install~.
• As I noted on #11496, you should copy the jmol shell script to SAGE_LOCAL/bin as part of spkg-install.
• It would be helpful if you could prepare the mercurial repository with the version of hg installed with Sage. When I use the version included with Sage (which is the only version on my computer), I get

  $ hg status
  abort: requirement 'dotencode' not supported!
  

  so I can't check the state of the repo without installing a new version of hg.

[gutow]

• Hmm...didn't notice the backup files.  I'll look into that...
• I could add the Jmol script to the jmol .spkg.  The one that is in there is the generic install not the one for SAGE...It seems reasonable that it should not be tracked by SAGE...I will add the SAGE version as a patch to the .spkg.  Once I get this done, I believe #11496 should be closed, as it will no longer apply...let me know who to notify about that.
• Not sure if I can manage to successfully backtrack on the Hg repository format.  In order to work with the flask-notebook on my machine, I found I needed a system install of Hg and I got the latest.  Is this going to be a hold up?  Hg has been a real pain compared to the other version control systems I've  used.  Can somebody point out how to do the backtrack cleanly and quickly.  If it takes me a couple of hours to untangle Hg that's more time than it will take me to fix the rest of the .spkg problems and more time than I have for this project at the moment.

Replying to jhpalmieri:

Several comments: - There are some backup files present: SPKG.txt~ and spkg-install~. - As I noted on #11496, you should copy the jmol shell script to SAGE_LOCAL/bin as part of spkg-install. - It would be helpful if you could prepare the mercurial repository with the version of hg installed with Sage. When I use the version included with Sage (which is the only version on my computer), I get $ hg status abort: requirement 'dotencode' not supported! so I can't check the state of the repo without installing a new version of hg.

[gutow]

.p2 addresses the following: * reverted to mercurial version included in Sage (lost all the history in the process).

• jmol launch script is now included in the patches and copied to SAGE_LOCAL/bin.
• I think I removed all the backup files.

[jhpalmieri]

I upgraded my version of mercurial so I can look at the repo, and I hope that the version included with Sage will be upgraded soon, too (see #10594). Oh, and now I see that I can open this repo with the old version. Anyway, there are some uncommitted files:

$ hg status
? patches/Jmol.js.patch
? patches/jmol.patch

(So you should run "hg add" and then commit again.)

The file spkg-install needs some work. In particular, any variable which refers to a path should be quoted, in case there are spaces in it. (Right now, we don't support having spaces in the path where Sage is, but when it's easy to allow, we should.) I think this should do it:

spkg-install

@@ -27 +27 @@
 fi
 
 # Check for sagenb location
-cd $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol"
+cd "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol"
 if [ $? -ne 0 ]; then
     echo "No old Jmol install in notebook. Skipping removal of Jmol from notebook."
 else
     echo "Removing Jmol files from the notebook data directory..."
-    rm -r $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol"
+    rm -r "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol"
 fi
 
 TEMPDIR=$SPKDIR"/src"
@@ -47 +47 @@
    echo "Directory "$SAGE_LOCAL"/share/jmol does not exist.  Creating Directory..."
 else
     echo "Deleting all files from "$SAGE_LOCAL"/share/jmol..."
-    rm -r $SAGE_LOCAL"/share/jmol"
+    rm -r "$SAGE_LOCAL/share/jmol"
     echo "replacing jmol directory and contents..."
 fi
 
-mkdir $SAGE_LOCAL"/share/jmol"
+mkdir "$SAGE_LOCAL/share/jmol"
 
 TEMPDIR=$SPKDIR"/src/jmol"
 cd "$TEMPDIR"
-cp -r * $SAGE_LOCAL"/share/jmol/"
+cp -r * "$SAGE_LOCAL/share/jmol/"
 
 cd $SAGE_LOCAL"/bin"
 if [ $? -ne 0 ]; then
@@ -64 +64 @@
 else
     echo "Copying jmol script to "$SAGE_LOCAL"/bin."
     cd "$TEMPDIR"
-    cp -f jmol $SAGE_LOCAL"/bin"
+    cp -f jmol "$SAGE_LOCAL/bin"
 fi
 
 echo "Installing applet web directory"
-mkdir $SAGE_LOCAL"/share/jmol/appletweb"
-cp Jmol.js $SAGE_LOCAL"/share/jmol/appletweb"
+mkdir "$SAGE_LOCAL/share/jmol/appletweb"
+cp Jmol.js "$SAGE_LOCAL/share/jmol/appletweb"
 
 TEMPDIR=$SPKDIR"/patches/appletweb"
 cd "$TEMPDIR"
@@ -77 +77 @@
    echo "Error finding patches/appletweb directory. Exiting."
    exit 1
 fi
-cp -r * $SAGE_LOCAL"/share/jmol/appletweb"
+cp -r * "$SAGE_LOCAL/share/jmol/appletweb"
 
 if [ $? -ne 0 ]; then
    echo "Error installing PACKAGE_NAME."

[gutow]

The below is done.  Try downloading the spkg again after 3 PM CDT.

Replying to jhpalmieri:

I upgraded my version of mercurial so I can look at the repo, and I hope that the version included with Sage will be upgraded soon, too (see #10594). Oh, and now I see that I can open this repo with the old version. Anyway, there are some uncommitted files: $ hg status ? patches/Jmol.js.patch ? patches/jmol.patch (So you should run "hg add" and then commit again.) The file spkg-install needs some work. In particular, any variable which refers to a path should be quoted, in case there are spaces in it. (Right now, we don't support having spaces in the path where Sage is, but when it's easy to allow, we should.) I think this should do it: #!diff diff --git a/spkg-install b/spkg-install --- a/spkg-install +++ b/spkg-install @@ -27,12 +27,12 @@ if [ $? -ne 0 ]; then fi # Check for sagenb location -cd $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol" +cd "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol" if [ $? -ne 0 ]; then echo "No old Jmol install in notebook. Skipping removal of Jmol from notebook." else echo "Removing Jmol files from the notebook data directory..." - rm -r $SAGE_ROOT"/devel/sagenb/sagenb/data/jmol" + rm -r "$SAGE_ROOT/devel/sagenb/sagenb/data/jmol" fi TEMPDIR=$SPKDIR"/src" @@ -47,15 +47,15 @@ if [ $? -ne 0 ]; then echo "Directory "$SAGE_LOCAL"/share/jmol does not exist. Creating Directory..." else echo "Deleting all files from "$SAGE_LOCAL"/share/jmol..." - rm -r $SAGE_LOCAL"/share/jmol" + rm -r "$SAGE_LOCAL/share/jmol" echo "replacing jmol directory and contents..." fi -mkdir $SAGE_LOCAL"/share/jmol" +mkdir "$SAGE_LOCAL/share/jmol" TEMPDIR=$SPKDIR"/src/jmol" cd "$TEMPDIR" -cp -r * $SAGE_LOCAL"/share/jmol/" +cp -r * "$SAGE_LOCAL/share/jmol/" cd $SAGE_LOCAL"/bin" if [ $? -ne 0 ]; then @@ -64,12 +64,12 @@ if [ $? -ne 0 ]; then else echo "Copying jmol script to "$SAGE_LOCAL"/bin." cd "$TEMPDIR" - cp -f jmol $SAGE_LOCAL"/bin" + cp -f jmol "$SAGE_LOCAL/bin" fi echo "Installing applet web directory" -mkdir $SAGE_LOCAL"/share/jmol/appletweb" -cp Jmol.js $SAGE_LOCAL"/share/jmol/appletweb" +mkdir "$SAGE_LOCAL/share/jmol/appletweb" +cp Jmol.js "$SAGE_LOCAL/share/jmol/appletweb" TEMPDIR=$SPKDIR"/patches/appletweb" cd "$TEMPDIR" @@ -77,7 +77,7 @@ if [ $? -ne 0 ]; then echo "Error finding patches/appletweb directory. Exiting." exit 1 fi -cp -r * $SAGE_LOCAL"/share/jmol/appletweb" +cp -r * "$SAGE_LOCAL/share/jmol/appletweb" if [ $? -ne 0 ]; then echo "Error installing PACKAGE_NAME."

[gutow]

Removing command line patch as it appears to be incorporated into 4.8.

[ddrake]

I'm looking this over, and I have some more minor nits to pick with spkg-install. Here's an idiom I see a couple times:

TEMPDIR=something
cd "$TEMPDIR"
do some stuff
...

Two things bother me about that: (1) you call it TEMPDIR, which implies to me that it's a temporary directory used for some scratch work, but you're talking about an existing directory with important files in it. (2) You never use the TEMPDIR variable, except in the cd command...so why not just cd to the directory?

Also, I think it's more readable to just put the paths into commands, rather than cding around. I would prefer to see

cp $SAGE_FOO/dir/file.txt $SAGE_BAR/dir/dir2/

rather than

cd $SAGE_FOO/dir
cp file.txt $SAGE_BAR/dir/dir2/

It makes it easier to read the script, since you don't have to remember what the working directory is. Also, you sometimes change directories inside an if-then-else, so the working directory depends on what happened there!

Finally, at the end of spkg-install, you have "PACKAGE NAME" instead of Jmol!

[ddrake]

changes to spkg-install, SPKG.txt. For review only.

[ddrake]

I've looked over the spkg, and modulo my fixes to spkg-install, I can give it a positive review. My changes are in attachment:trac11503.patch and I have a spkg with those changes at  http://sagenb.kaist.ac.kr/~drake/Jmol-12.0.45.p3.spkg. It's identical to Jonathan's version except for the changes in attachment:trac11503.patch .

Jason, you volunteered to look it over, but that shouldn't stop anyone else. :)

[gutow]

Dan's changes look good to me (obviously much more experience writing these scripts than I have:)) I suggest we try Dan's .spkg for the review and have changed the description to reflect this.

Jonathan

Replying to ddrake:

I've looked over the spkg, and modulo my fixes to spkg-install, I can give it a positive review. My changes are in attachment:trac11503.patch and I have a spkg with those changes at  http://sagenb.kaist.ac.kr/~drake/Jmol-12.0.45.p3.spkg. It's identical to Jonathan's version except for the changes in attachment:trac11503.patch . Jason, you volunteered to look it over, but that shouldn't stop anyone else. :)

[jason]

Dan: your changes look fine to me. Do you give everything else a positive review?

[jason]

Oh, wait, I see that you just said modulo your fixes, you give the spkg a positive review. So it sounds like we agree: positive review.

[jason]

hmm...nothing has been added to the deps file. There should be another patch for that.

[jason]

So I think what needs to be done at this point is:

1. Make the jmol spkg not start with a capital letter (simple rename of the spkg file)

2. Apply the trac-11503-jmol-spkg.patch to the sage root repository, make an sdist, put the jmol spkg in there, and try building. In essence, review the changes to the sage root repository.

[jdemeyer]

For SPKG.txt, you should follow the format at  http://sagemath.org/doc/developer/producing_spkgs.html#creating-a-new-spkg

[jdemeyer]

Okay, it's not in the documentation, but you should format the Changelog with lines like

=== mypackage-3.18.spkg (Jeroen Demeyer, 17 January 2012) ===
 * Changed foo to bar

This is how every spkg does it (except for sagenb, which I can live with).

[gutow]

Should I change this or will you Dan?

A comment.  As a relatively newer developer, I do not appreciate formatting requirements that are not documented.  I followed the example in the developer's guide.

Jonathan

Replying to jdemeyer:

Okay, it's not in the documentation, but you should format the Changelog with lines like === mypackage-3.18.spkg (Jeroen Demeyer, 17 January 2012) === * Changed foo to bar This is how every spkg does it (except for sagenb, which I can live with).

[jason]

I made an spkg that reformats the changelog and makes the spkg lowercase. See the new instructions.

Jonathan: I understand---I didn't object before because it wasn't in the docs.

[jdemeyer]

Yeah, I can imagine it's annoying. It should be documented better. I always thought it was documented, for the simple reason that every spkg is doing it the same way.

The docs should probably be clarified.

[jdemeyer]

I made some further improvements to SPKG.txt, in particular I simplified the Dependencies to

 * Sage Notebook (sagenb)

[jason]

I've updated the instructions over at #11080 to reflect your new spkg.

[jhpalmieri]

Replying to jdemeyer:

Yeah, I can imagine it's annoying. It should be documented better.

See  http://trac.sagemath.org/sage_trac/attachment/ticket/9419/trac_9419-part2.patch.

[kcrisman]

Yeah, I can imagine it's annoying. It should be documented better.

See  http://trac.sagemath.org/sage_trac/attachment/ticket/9419/trac_9419-part2.patch.

I knew this sounded familiar!

[jason]

apply to sage repository

[jason]

Attached patch which fixes command-line jmol. This issue was discovered over on #9238; I used the fix from the comments there.

[jason]

To review this ticket, this is what is left:

• My and jdemeyer's changes to the spkg (reformatting SPKG.txt, basically)
• the trac-11503-jmol-commandline.patch that fixes command-line jmol

[jdemeyer]

Since jmol doesn't use setuptools, it should be listed normally between the dependencies. My new trac-11503-jmol-spkg.patch does this.

[kini]

Jason, what did you base your commandline patch on? It doesn't apply to 4.8.rc0, even with fuzz, since there seem to be some different lines in the context of the second chunk. Here's a rebase on 4.8.rc0, though I don't know what's going on here so I won't put it in the ticket description.

[kini]

apply to $SAGE_ROOT/devel/sage

[jason]

I made the patch on 5.0something (prealpha/beta1 maybe?). I also had #11078 applied before this.

[kini]

Oh, OK. Then there is no rebasing needed -- 4.8.rc0 -> #11078 -> your patch works perfectly.

[jason]

I tried building a clean sage with this spkg and the new notebook, and command-line jmol didn't work because the local/bin/jmol script was not executable. So it should probably be set to executable inside the spkg.

[jason]

I just uploaded a p4 version that sets the jmol script to executable after copying to local/bin. So it's needs review again.

[jdemeyer]

apply to the sage root repository

[ppurka]

Rebased to sage-5.0beta11

[ppurka]

The attachment trac-11503-jmol-spkg.2.patch has been rebased to sage-5.0beta11. Compiling it now (along with #11874 and the patches to SAGE_ROOT from #11080).

[gutow]

Updated to Jmol-12.2.21...this will allow using Jmol to generate images server-side.

[jdemeyer]

$ hg status
? SPKG.txt~

[kini]

Since it's a new jmol version, the p-version number should be reset.

[gutow]

Odd that the chmod is gone...I didn't edit the script....All I did was replace the .jar files....I'll check this afternoon.  Not sure why the SPKG backup got left behind, but I can delete it....

Replying to jdemeyer:

$ hg status ? SPKG.txt~

[gutow]

SPKG.txt~ removed and renamed to ...p0

[gutow]

I believe this ticket is actually included in the positively reviewed #11080.  If there are no objections, I believe this should become a positive review as well.

[ppurka]

Indeed this should be positive review. I have been using it both from the notebook and from the command line without issues, for several months.

Setting it to positive review.

[kini]

Please put the latest version of the SPKG up on this ticket. It looks like #12299's instructions mention a newer version of the SPKG than is found here.

[kcrisman]

Jonathan can correct me if I'm wrong, but as far as I know, this is on purpose, because the newest Jmol at #12299 needs the many updates to Jmol there. In that sense this ticket is a 'hard' prereq for #11080 and hence for #13121/#12299.

[kini]

OK, fair enough.

[jdemeyer]

Unmerging this from sage-5.2 due to the serious security issue at #13270.