Opened 9 years ago

Last modified 8 years ago

#16004 closed enhancement

Update notebook to utilize pure javascript JSmol for default live 3-D — at Version 40

Reported by: Jonathan Gutow Owned by: Jonathan Gutow
Priority: major Milestone: sage-6.4
Component: notebook Keywords: Jmol 3D 3-D
Cc: Karl-Dieter Crisman, Andrey Novoseltsev, Steven Trogdon, Jason Grout, Frédéric Chapoton Merged in:
Authors: gutow Reviewers:
Report Upstream: Reported upstream. Developers acknowledge bug. Work issues:
Branch: Commit:
Dependencies: #16003 Stopgaps:

Status badges

Description (last modified by Jonathan Gutow)

The notebook needs updates to utilize a pure javascript version of Jmol by default for live 3-D. This includes changes to make the notebook compatible with the newer jquery-1.9 (some things used in the notebook have been deprecated).

Jmol is cross compiled to java and javascript. Because so many browser/OS combinations now actively try to avoid using java applets, it is necessary to have a pure javascript alternative. This solution provides all the functionality of Jmol, but as expected since javascript is interpreted rather than bytecode ("~3/4" of the way to compiled) the performance when interacting with the plot is slower.

This first implementation removes the user tools that have appeared to the right of the 3-D plot as they need to be rewritten as well. In this first implementation it is not possible to choose to use the java applet instead.

One added feature is a "load 3-D live" check box that can be used to make a worksheet load 3-D plots live when the worksheet is opened. For worksheets with a single interact or only 1-2 plots this should be OK.

*This requires installation of an updated Jmol.spkg. See ticket #16003.

Sorry for the github commit, but I could not get a notebook branch working in Sage.

How to test:

  1. Clone the git repository https://github.com/gutow/sagenb to anywhere convenient on your system.
  2. Do the following to make sage use the new repository.
    $cd SAGE_ROOT/local/lib/python/site-packages/sagenb-0.10.8.2-py2.7.egg
    $mv sagenb sagenb_old
    $ln -s <path to the sagenb directory inside your new repository> sagenb 
    

In my case the symlink looks like ln -s ~/git/sagenb/sagenb sagenb

Change History (40)

comment:1 Changed 9 years ago by Jonathan Gutow

Component: PLEASE CHANGEnotebook
Description: modified (diff)

comment:2 Changed 9 years ago by Jonathan Gutow

Owner: set to Jonathan Gutow

comment:3 Changed 9 years ago by Jonathan Gutow

Description: modified (diff)

comment:4 Changed 9 years ago by Jonathan Gutow

Description: modified (diff)

comment:5 Changed 9 years ago by Jonathan Gutow

Description: modified (diff)
Status: newneeds_review

comment:6 Changed 9 years ago by Karl-Dieter Crisman

Dependencies: 16003#16003

comment:7 Changed 9 years ago by Karl-Dieter Crisman

Cc: Karl-Dieter Crisman Andrey Novoseltsev Steven Trogdon added; krisman nosvelt strongdon removed

comment:8 Changed 9 years ago by Karl-Dieter Crisman

Description: modified (diff)

comment:9 Changed 9 years ago by Jonathan Gutow

I'm not sure it has to do with this or something broken in my jquery.form.min.js installation, but I can no longer rename worksheets. Does anybody else seeing this or have an idea how this could have anything to do with the changes in this ticket?

comment:10 in reply to:  9 Changed 9 years ago by Jonathan Gutow

Replying to gutow:

I'm not sure it has to do with this or something broken in my jquery.form.min.js installation, but I can no longer rename worksheets. Does anybody else seeing this or have an idea how this could have anything to do with the changes in this ticket?

I think I've found it. This may be painful. The newer jquery seems to be killing stuff in jquery-ui-1.8.11.custom.js. I will look for a version that is compatible with the newer jquery.

comment:11 Changed 9 years ago by Jonathan Gutow

jquery-ui is definitely out-of-date. The 1.8.11 release is from 2011. Does anybody know what selections need to be made to get the theme for the notebook to look right? I think to get everything working will require updating to the latest 1.10.4.

comment:12 Changed 9 years ago by Andrey Novoseltsev

Cc: Jason Grout added

First of all, thanks for all your work on this, Jonathan!

And perhaps Jason can provide some insight in jquery problems?

comment:13 Changed 9 years ago by Jason Grout

See my message to sage-devel: https://groups.google.com/d/msg/sage-devel/8zS6qofxxwk/K5LcLsCWBeMJ for the configuration we use in the cell server with jquery-ui 1.10.something.

comment:15 Changed 9 years ago by Jonathan Gutow

Description: modified (diff)

comment:16 in reply to:  15 Changed 9 years ago by Jonathan Gutow

Replying to gutow: With Jason's directions for building the theme and a little tweaking, I believe I have everything working again along with the javascript 3-D. As a bonus we now how little icons to click on for adding compute and rich-text cells, so forgetful people like me do not have to remember which key-click combinations to use. The real reason for the icons is that I could not get the key combinations to work:)

Last edited 9 years ago by Jonathan Gutow (previous) (diff)

comment:17 Changed 8 years ago by Jonathan Gutow

Status: needs_reviewneeds_work

Some jquery-ui image paths broken (probably issue of symlinks)...

comment:18 Changed 8 years ago by Karl-Dieter Crisman

Hey, just a quick question as I hopefully will be able to get testing this soon - can the 'load 3-D live" checkbox work just as well with jmol as with jsmol? Also, I presume it will be evident once I finally get it going, but how does one get the Jmol instead of jsmol (say, locally, so that things are snappier)?

comment:19 Changed 8 years ago by Karl-Dieter Crisman

Description: modified (diff)

comment:20 in reply to:  17 Changed 8 years ago by Karl-Dieter Crisman

Some jquery-ui image paths broken (probably issue of symlinks)...

Is this fixed in https://github.com/gutow/sagenb/commit/7aacbfa3957f237edbea448d492f7c2b7d34cec2 ?

comment:21 Changed 8 years ago by Karl-Dieter Crisman

I think that creating a new branch inside sagenb is now not possible with the new layout (not directly git-related). One has to even figure out where sagenb lives... annoying. So to test this one really needs a new sagenb spkg to download, or create one. Harrumph.

comment:22 Changed 8 years ago by Steven Trogdon

Unless I've missed something it would seem that a new sagenb spkg is needed. The problem is the upstream sagenb-0.10.8.2 tarball is a tarball of tarballs. And it is the sagenb sub-tarball that has to be modified. Now I was able to generate patches that could be applied directly to the local/lib/python2.7/site-packages/sagenb-0.10.8.2-py2.7.egg folder by cloning https://github.com/gutow/sagenb. It seems that there are a couple more commits needed than listed above. But generating these patches is a lot of work! I'm able to see the icons and to turn on 3-D which is very, very slow. However, I see no way to evaluate an individual cell.

comment:23 Changed 8 years ago by Karl-Dieter Crisman

Actually, ppurka has a good solution for this at this sage-devel thread. At least I assume it works - I can't check right this second.

$ cd ~/sagenb/sagenb
$ rm flask_version
$ mv ../flask_version .
$ cd ~/sage/local/lib/python/site-packages/sagenb-0.10.8.2-py2.7.egg
$ mv sagenb{,-0.10.8.2}
$ ln -s ~/sagenb/sagenb .

comment:24 Changed 8 years ago by Jonathan Gutow

Description: modified (diff)

I'm updating the instructions above to include a fix that leaves the sagenb egg unchanged and allows you to work with multiple notebook development repositories.

To answer another question above the very latest commit to the repository does include the path fixes for jquery-ui images.

comment:25 Changed 8 years ago by Jonathan Gutow

Description: modified (diff)

comment:26 in reply to:  18 ; Changed 8 years ago by Jonathan Gutow

At this point I have not got the option to run the java applet working cleanly. I can do it by manually screwing with things, but I am still working on it. I have not figured out how to modify the templates so that it will not use java applets without the user specifically requesting it. Because of how evil certain browsers/OS combinations have got about using java applets, we do not want that to ever be the default, even though it is better.

Replying to kcrisman:

Hey, just a quick question as I hopefully will be able to get testing this soon - can the 'load 3-D live" checkbox work just as well with jmol as with jsmol? Also, I presume it will be evident once I finally get it going, but how does one get the Jmol instead of jsmol (say, locally, so that things are snappier)?

comment:27 in reply to:  26 Changed 8 years ago by Karl-Dieter Crisman

Replying to gutow:

At this point I have not got the option to run the java applet working cleanly. I can do it by manually screwing with things, but I am still working on it. I have not figured out how to modify the templates so that it will not use java applets without the user specifically requesting it. Because of how evil certain browsers/OS combinations have got about using java applets, we do not want that to ever be the default, even though it is better.

Well, for instance, could one just hack into Sage itself... after all, one would want to be able to set viewer= whatever, including both jsmol and jmol. What happens with viewer='jmol' with your branch? (I'm trying to set it up but this is an incredibly busy week on campus for me, so I'm not sure how much progress I'll make.) At the least, one would want to just be able to add this to plots to get the Java.

In fact,

sage: from sage.plot.plot3d.base import SHOW_DEFAULTS
sage: SHOW_DEFAULTS['viewer'] = 'jmol'

should be possible then. But I don't know whether that would tie in with your changes thus far.

Last edited 8 years ago by Karl-Dieter Crisman (previous) (diff)

comment:28 Changed 8 years ago by Jonathan Gutow

Description: modified (diff)
Status: needs_workneeds_review

All seems to work on my test server. I need some reviewers testers now.

comment:29 Changed 8 years ago by Karl-Dieter Crisman

Description: modified (diff)

comment:30 Changed 8 years ago by Karl-Dieter Crisman

I think it should be

$ln -s <path to the sagenb directory inside your new repository> sagenb 

not

$ln -s sagenb <path to the sagenb directory inside your new repository> 

am I right?

comment:31 Changed 8 years ago by Karl-Dieter Crisman

Description: modified (diff)

Okay, I got it working. Preliminary thoughts:

  • Snappier than advertised. But SO SLOW with lots of points; a sparse vector field or basic plot is not so horrible. Of course, so was the Java version...
  • The load 3d live button is precisely what the doctor ordered.
  • Is it possible that inside interacts the load 3d live button doesn't work (that is, interacts are always live?) Or is it possible that you can only turn that button on, but not off (counter the expectation when clicking it)? I observed some unusual behavior, but maybe it was just something dumb I did.
  • How do you turn a plot "off" if there are too many open? (Or is that not a problem like it was with the sleeping Java applets?)
  • I see no way to get the things I used to with right-click (like stereo glasses, color, axes, whatever). Does jsmol just not have that?
  • The icon for activating the plot is ... not obvious. I still really find the auto-disable piece annoying, and I know that some people have just hacked that away, though I also know I won't convince you otherwise ;-)
  • Please remove the icons for new cells. They are cute, and probably an improvement in some ways, but it should really be orthogonal to this issue. There's enough moving parts to think about as it is. Especially since the blue bars have been the way to make new cells for many years! That's a pretty big change.
  • I figure out what strogdon is talking about. Try activating a jsmol, and then move it around a bit, and then try clicking in a cell *without* clicking alllll the way on the right to "lose focus" from the jsmol. (This is a problem anyway, of having to go all the way to the right to scroll - not limited to jsmol, though, threejs in the cell server has this problem as well.) You won't be able to, at least often not; you have to somehow get off focus and there is some timing delay.
  • I also haven't tried all the zillions of options out there for 3d plots to see if it still obeys them all, but presumably if it's pretty similar to jmol it will do fine.
  • Does it require WebGL turned on? I think not, but I'm not sure.

Hope this helps! Actually, it's fairly comparable to the current setup, so probably we could get it in sooner rather than later. Especially if you have any thoughts on the "viewer" thing - to me, this would be by far the easiest way to switch back and forth between functionality.

Once this has been tested enough and the "right" commits used, I think that you can just do a pull request to sagenb on github with your branch for this. Yay! Thanks for what is clearly very good hard work.

Last edited 8 years ago by Karl-Dieter Crisman (previous) (diff)

comment:32 in reply to:  31 ; Changed 8 years ago by Jonathan Gutow

Replying to kcrisman:

Okay, I got it working. Preliminary thoughts:

  • Snappier than advertised. But SO SLOW with lots of points; a sparse vector field or basic plot is not so horrible. Of course, so was the Java version...
  • The load 3d live button is precisely what the doctor ordered.

Thought this is what people were asking for.

  • Is it possible that inside interacts the load 3d live button doesn't work (that is, interacts are always live?) Or is it possible that you can only turn that button on, but not off (counter the expectation when clicking it)? I observed some unusual behavior, but maybe it was just something dumb I did.

I believe I wrote the code so that if that box is checked all Jmol/JSmols will load live. If it is not checked they should not load live. I am not aware of anything interacts do that should force a live load.

  • How do you turn a plot "off" if there are too many open? (Or is that not a problem like it was with the sleeping Java applets?)

Turning them off is presently not possible. For the javascript version (JSmol) I do not think it will matter as long as you do not have the ones you are not interacting with doing things that require continuous computations like spinning or running an animation (something Jmol/JSmol can do, but Sage presently does not generate the input for).

  • I see no way to get the things I used to with right-click (like stereo glasses, color, axes, whatever). Does jsmol just not have that?

Right-click still brings up a menu. For the time being I've left the default Jmol menu as it provides access to some file writing features. You were not able to get it? I'll look into that.

  • The icon for activating the plot is ... not obvious. I still really find the auto-disable piece annoying, and I know that some people have just hacked that away, though I also know I won't convince you otherwise ;-)

It is just the default image (a standard stock "play" icon) stored in the JSmol tree. If we can come up with something better it can be replaced.

  • Please remove the icons for new cells. They are cute, and probably an improvement in some ways, but it should really be orthogonal to this issue. There's enough moving parts to think about as it is. Especially since the blue bars have been the way to make new cells for many years! That's a pretty big change.

I agree that it would be nice to separate this "big" change in the interface out. Unfortunately, I really could not get the old key+click actions to work with the new jquery that is required to support JSmol. Unless somebody else can make it work without breaking JSmol these icons are going to be part of this change. I'm open to suggestions on exactly how things should look, but built it using the standard jquery-ui tools that are part of sagenb.

  • I figure out what strogdon is talking about. Try activating a jsmol, and then move it around a bit, and then try clicking in a cell *without* clicking alllll the way on the right to "lose focus" from the jsmol. (This is a problem anyway, of having to go all the way to the right to scroll - not limited to jsmol, though, threejs in the cell server has this problem as well.) You won't be able to, at least often not; you have to somehow get off focus and there is some timing delay.

I'll play with this. Haven't noticed it, at least not anymore than most GUI based software.

  • I also haven't tried all the zillions of options out there for 3d plots to see if it still obeys them all, but presumably if it's pretty similar to jmol it will do fine.

If it worked in Jmol it works in JSmol as JSmol is just just cross compiled to javascript. There have been some minor oddities, but the demands of the X-ray crystallography users are much more severe than what is usually being done here. I think Bob Hanson and the others working on the conversion between Java and Javascript have found most of problems by now. The chemistry community has been replacing java applet only versions with this for 9-12 months now.

  • Does it require WebGL turned on? I think not, but I'm not sure.

I have set it up to use HTML5 only. WebGL support exists, but has problems with transparency and other such things.

Hope this helps! Actually, it's fairly comparable to the current setup, so probably we could get it in sooner rather than later. Especially if you have any thoughts on the "viewer" thing - to me, this would be by far the easiest way to switch back and forth between functionality.

I'll keep working on this. I think if I can get it working cleanly in the GUI, it will be easy to couple it to the viewer option.

Once this has been tested enough and the "right" commits used, I think that you can just do a pull request to sagenb on github with your branch for this. Yay! Thanks for what is clearly very good hard work.

comment:33 in reply to:  32 ; Changed 8 years ago by Karl-Dieter Crisman

  • The load 3d live button is precisely what the doctor ordered.

Thought this is what people were asking for.

Yes, absolutely!

  • I see no way to get the things I used to with right-click (like stereo glasses, color, axes, whatever). Does jsmol just not have that?

Right-click still brings up a menu. For the time being I've left the default Jmol menu as it provides access to some file writing features. You were not able to get it? I'll look into that.

Yeah. This is on Safari 5, granted, but still... actually, it doesn't work on FF either. I don't know why, other right-clicking seems to be working fine on this computer still.

  • The icon for activating the plot is ... not obvious. I still really find the auto-disable piece annoying, and I know that some people have just hacked that away, though I also know I won't convince you otherwise ;-)

It is just the default image (a standard stock "play" icon) stored in the JSmol tree. If we can come up with something better it can be replaced.

I think that a word of some kind would be helpful. The image doesn't say much to me, while your previous "live image" or whatever made it very clear. Not everyone will mouse over that icon.

  • Please remove the icons for new cells. They are cute, and probably an improvement in some ways, but it should really be orthogonal to this issue. There's enough moving parts to think about as it is. Especially since the blue bars have been the way to make new cells for many years! That's a pretty big change.

I agree that it would be nice to separate this "big" change in the interface out. Unfortunately, I really could not get the old key+click actions to work with the new jquery that is required to support JSmol. Unless somebody else can make it work without breaking JSmol these icons are going to be part of this change. I'm open to suggestions on exactly how things should look, but built it using the standard jquery-ui tools that are part of sagenb.

Huh, that is weird. When I start actually reviewing the code changes I'll try to think about that.

Hope this helps! Actually, it's fairly comparable to the current setup, so probably we could get it in sooner rather than later. Especially if you have any thoughts on the "viewer" thing - to me, this would be by far the easiest way to switch back and forth between functionality.

I'll keep working on this. I think if I can get it working cleanly in the GUI, it will be easy to couple it to the viewer option.

That would be very good, providing all available options.

comment:34 in reply to:  33 ; Changed 8 years ago by Jonathan Gutow

Status: needs_reviewneeds_work

Replying to kcrisman:

  • I see no way to get the things I used to with right-click (like stereo glasses, color, axes, whatever). Does jsmol just not have that?

Right-click still brings up a menu. For the time being I've left the default Jmol menu as it provides access to some file writing features. You were not able to get it? I'll look into that.

Found the problem, but it's going to be a toughy...relates to the load order of the various js files. It appears a function in jquery clobbers a JSmol function or vice-versa if we switch the load order of the two. I can either get the right-click JSmol menu working or dialogs in Sagenb. I'm consulting with the Jmol/JSmol community on this.

  • The icon for activating the plot is ... not obvious. I still really find the auto-disable piece annoying, and I know that some people have just hacked that away, though I also know I won't convince you otherwise ;-)

It is just the default image (a standard stock "play" icon) stored in the JSmol tree. If we can come up with something better it can be replaced.

I think that a word of some kind would be helpful. The image doesn't say much to me, while your previous "live image" or whatever made it very clear. Not everyone will mouse over that icon.

Actually, I think the text pops up if you mouse over the whole image. But I see your point. The idea was to make it "youtube-like".

comment:35 in reply to:  34 ; Changed 8 years ago by Karl-Dieter Crisman

  • I see no way to get the things I used to with right-click (like stereo glasses, color, axes, whatever). Does jsmol just not have that?

Right-click still brings up a menu. For the time being I've left the default Jmol menu as it provides access to some file writing features. You were not able to get it? I'll look into that.

Found the problem, but it's going to be a toughy...relates to the load order of the various js files. It appears a function in jquery clobbers a JSmol function or vice-versa if we switch the load order of the two. I can either get the right-click JSmol menu working or dialogs in Sagenb. I'm consulting with the Jmol/JSmol community on this.

Hmm, so the functions happen to be named the same thing?


Okay, I think that if we can fix the following in one way or another, and it otherwise it seems to handle things well, we can probably move to this:

  • Fix the clobbered function you mention above.
  • Fix or somehow make the icon here more obvious:

    Actually, I think the text pops up if you mouse over the whole image. But I see your point. The idea was to make it "youtube-like".

  • Find some resolution to the new versus old ways to get new cells, ideally using the old clicking syntax (else a major change in interface).
  • Getting viewer to work with this so we can provide jmol or jsmol.

I know dealing with these is a lot harder than it seems to me, so hopefully we can find something good. I'm going to also look into the shift-click thing for a minute in the hopes I get lucky...

Random request: In this commit it would be helpful to know which of this truly huge commit is just adding the new jquery, and what is something for Sage. There are occasional changes I couldn't figure out if they were not important or if they were quite important, and it is kind of a bomb, by necessity. So any hints on what to focus on there would be very helpful.

comment:36 in reply to:  35 Changed 8 years ago by Karl-Dieter Crisman

  • Find some resolution to the new versus old ways to get new cells, ideally using the old clicking syntax (else a major change in

So it seems that this was just some custom thing written way back when.

Currently this stuff is in sagenb/data/jquery/plugins/extendedclick. In fact, it seems to still be there after the update as well, and it's still loaded in base.html. So maybe we can just update it for the new jquery after all...

In fact, Jason has a github repo for this! At the time he updated for a new jquery too. I did try editing things back in Safari's Firebug equivalent, but any clicking at all - even in the compute cells - yielded

TypeError: 'undefined' is not an object (evaluating 'introspect[id].before_replacing_word')

from line 23 of master.js. Not sure if this is related. Anyway, I would be very surprised if one couldn't fix this but I couldn't get the web inspector to allow me to change any of the js files, very annoying.

Or, we could perhaps use this jquery plugin or this javascript thing instead? This link seems less robust but could work too...

Last edited 8 years ago by Karl-Dieter Crisman (previous) (diff)

comment:37 Changed 8 years ago by Jonathan Gutow

Random request: In ​this commit it would be helpful to know which of this truly huge commit is just adding the new jquery, and what is something for Sage. There are occasional changes I couldn't figure out if they were not important or if they were quite important, and it is kind of a bomb, by necessity. So any hints on what to focus on there would be very helpful.


I couldn't seem to update the commit message without redoing the commit. So here's the answer. That commit is just replacing all the old jquery and jquery-ui files. Nothing Sage related. I wonder if we should start handling things like that as upstream packages? A ticket in itself?

Still testing, but believe I have solved the dialog/JSmol popup conflict. There is an alternative JSmol package to avoid these kinds of problems...

Still no luck on getting the cntl-click stuff to work. I suspect it simply will require a careful rewrite of the old code. I'm not convinced that is a good use of anybody's time as we have a functional (and for novices more usable) alternative.

comment:38 Changed 8 years ago by Andrey Novoseltsev

Just to clarify - the problems are only with the mouse use for cells, keyboard commands like Ctrl+Enter, Alt+Enter etc. are still functional, correct?

comment:39 Changed 8 years ago by Karl-Dieter Crisman

Still no luck on getting the cntl-click stuff to work. I suspect it simply will require a careful rewrite of the old code. I'm not convinced that is a good use of anybody's time as we have a functional (and for novices more usable) alternative.

But it's backwards-incompatible, and requires rewriting of the help pages, and so forth. (The page that is linked to "Help" in the notebook.)

Just to clarify - the problems are only with the mouse use for cells, keyboard commands like Ctrl+Enter, Alt+Enter etc. are still functional, correct?

Yes, they appear to. But I don't think there is a way to get a text cell without the mouse - or is there? Also, sometimes one might not want to evaluate the current cell before inserting a new cell - and how would one get a new cell at the top of the page? Things to consider.

comment:40 Changed 8 years ago by Jonathan Gutow

Description: modified (diff)

OK. I think I've fixed the following with the latest commit:

  • Restored legacy click and shift-click behavior for adding cells to worksheets. This no longer uses the old code but is modern jquery. I have not taken out the old code because I believe it is used for keyboard key combinations as well. Can somebody double check me on this. If I'm wrong we can take it out.
  • Fixed the conflict between the dialog boxes and the Jmol/JSmol popup menu.

Still to do:

  • Change icons/wording so making JSmol live is more obvious.
  • See if I can fix defaults to use wire images during rotation for faster response.
  • Should I add back the link to help for the 3D viewer?

Can anybody explain in more detail what the strange behavior with the "load live" checkbox was? Karl-Dieter, Thanks for spotting this.

Note: See TracTickets for help on using tickets.