Opened 10 years ago

Last modified 8 months ago

#12402 closed enhancement

Make a three.js backend for 3d plotting — at Version 57

Reported by: jason Owned by: jason, was
Priority: major Milestone: sage-7.5
Component: graphics Keywords:
Cc: kini, novoselt, mbejger, gagern, egourgoulhon, paulmasson, yzh, mkoeppe Merged in:
Authors: Paul Masson Reviewers:
Report Upstream: N/A Work issues:
Branch: u/paulmasson/12402 (Commits, GitHub, GitLab) Commit: 62e7a8baaf9f4f193fbf1ff8b3406f2aa634d68d
Dependencies: Stopgaps:

Status badges

Change History (58)

comment:1 Changed 10 years ago by kini

  • Cc kini added

comment:2 Changed 10 years ago by novoselt

  • Cc novoselt added

comment:3 Changed 10 years ago by was

  • Description modified (diff)

comment:4 Changed 9 years ago by kcrisman

Jason, William, can you put whatever code exists on here since public worksheets are disabled? We don't want this just in the cloud.sagemath :-)

Changed 9 years ago by jason

comment:6 follow-up: Changed 9 years ago by kcrisman

Just as an update, this is now apparently working in cloud.sagemath.

comment:7 in reply to: ↑ 6 Changed 9 years ago by was

Replying to kcrisman:

Just as an update, this is now apparently working in cloud.sagemath.

It sort of works, but it isn't included by default. It will be soon though, so that anybody can at least test out some minimal functionality. I'll post when that happens.

comment:8 Changed 9 years ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:9 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:10 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:11 Changed 8 years ago by mbejger

  • Cc mbejger added

comment:12 follow-up: Changed 8 years ago by jason

Just FYI, there are now three different threejs backends: SMC, the threejs command in the sage cell, and the pythreejs-based backend that works in the sage cell too.

comment:13 Changed 8 years ago by novoselt

Jason - what is the status of the last one? Will it be pushed to get "standard" or there is still work to be done there and with your switch nobody is available to steer it?

The first two were definitely lacking features, in particular "things that don't scale with zoom" plus some transparency issues.

comment:14 follow-up: Changed 8 years ago by jason

The transparency issues are not fixed---those are inherent in how three.js draws things, and stem from limitations of webgl (there are experiements overcoming them in webgl, which might be interesting to look at, though; the author of mathbox is looking at them, for example).

I think that anything "pushed to standard" should be supported by SMC and the sage cell. Right now, the pythreejs solution requires an communication framework that is/emulates the ipython comm/widget framework. Something like that would need to be implemented for SMC to get it to work there. I think that would be a great project in itself, though---there is starting to be some good development of interactive things for the IPython notebook, and it would be great to use those things from SMC too.

comment:15 in reply to: ↑ 14 Changed 8 years ago by kcrisman

I think that anything "pushed to standard" should be supported by SMC and the sage cell. Right now, the pythreejs solution requires an communication framework that is/emulates the ipython comm/widget framework. Something like that would need to be implemented for SMC to get it to work there. I think that would be a great project in itself, though---there is starting to be some good development of interactive things for the IPython notebook, and it would be great to use those things from SMC too.

But I thought SMC only uses the js plotting - does it also do Jmols too? (You'll notice I still haven't had time to properly try out SMC, and won't for some time).


See also #16004 as a related ticket I also haven't had enough time to review properly.

comment:16 in reply to: ↑ 12 ; follow-up: Changed 8 years ago by mbejger

Replying to jason:

Just FYI, there are now three different threejs backends: SMC, the threejs command in the sage cell, and the pythreejs-based backend that works in the sage cell too.

Thanks :-) I am interested in enhancing the capabilities of SageManifolds? (#14865), which is still under development, so I guess the most natural thing would be to somehow use threejs locally. Probably this question was asked already many times, but what is the current status of threejs in sage notebook (local ipython notebook)?

comment:17 in reply to: ↑ 16 Changed 8 years ago by kcrisman

Thanks :-) I am interested in enhancing the capabilities of SageManifolds? (#14865), which is still under development, so I guess the most natural thing would be to somehow use threejs locally. Probably this question was asked already many times, but what is the current status of threejs in sage notebook (local ipython notebook)?

The Sagenb is not really at all an Ipython notebook, though it may have inspired earlier versions thereof. Anyway, the current status is that you can get it to work - see the examples above - but no one has put time into this. Partly because the sagenb is in maintenance mode currently (though it would be nice to have it more actively developed) but also because of the slight backwards functionality loss involved as you see in previous comments.

comment:18 Changed 8 years ago by gagern

  • Cc gagern added

comment:19 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:20 Changed 8 years ago by egourgoulhon

  • Cc egourgoulhon added

comment:21 Changed 6 years ago by paulmasson

  • Cc paulmasson added

comment:22 Changed 6 years ago by paulmasson

  • Branch set to u/paulmasson/12402

comment:23 Changed 6 years ago by paulmasson

  • Commit set to 7cedf75db69671cfb0a201435e5148d1041f0f05
  • Description modified (diff)
  • Milestone changed from sage-6.4 to sage-7.4
  • Status changed from new to needs_review

Here's some basic Three.js support for Graphics3d objects. It is modeled on the corresponding function in SageMathCell: simply use threejs() to wrap any combination of points, lines and surfaces. This will save an HTML file with the JavaScript needed to render the scene and open it in a browser.

There are a great many features that can be added in the future, but I first would like to see how much interest there is in this. There have been problems surfacing with Jmol (#20978 for example) and I certainly think it would be preferable to have a native viewer that can be updated more quickly. The HTML format is also more portable than Jmol files, since it only requires a modern browser to run.

This version relies on an external CDN to retrieve the Three.js library, since it does not currently download to the hard drive during the build process. I'll need help on figuring out how to change that at some point, since I'm sure people will want to be able to view saved files offline.


Last 10 new commits:

3d1cd2aHandle nested lists
efb3d0aHandle nested lists
f70ac3dAdd line plots
55f12e6Add point plots
b4c8414Adjust centering and camera
374809aRebase to move point bounds off points
9a42352Add auto z-aspect
d560683Minor rebase
aa322adAdd axis labels
7cedf75Check for equal bounds

comment:24 Changed 6 years ago by paulmasson

  • Authors set to Paul Masson

comment:25 follow-ups: Changed 6 years ago by egourgoulhon

Wow! I've just tried it and it's very nice! Thank you very much for this enhancement!

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

comment:26 in reply to: ↑ 25 ; follow-ups: Changed 6 years ago by kcrisman

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

See e.g. here - this would be a very good enhancement, though I guess it would have to then depend on the backend.

Can you think of anything it does not do? I seem to recall that there were things jsmol did with shading etc. that three.js couldn't, when this was discussed ~ 5 years ago.

comment:27 in reply to: ↑ 26 ; follow-up: Changed 6 years ago by egourgoulhon

Replying to kcrisman:

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

See e.g. here

Yes, set_axes_labels is the (not very satisfactory) workaround I wrote to deal with this when using Jmol ;-)

  • this would be a very good enhancement, though I guess it would have to then depend on the backend.

Maybe not: looking at the source code, in line 155 of src/sage/plot/plot3d/threejs.py, there is

addLabel( 'x=' + ( xMid ).toFixed(d).toString(), xMid, b[1].y+offset, a[2]*b[0].z );

It seems easy to replace 'x=' by another string.

Can you think of anything it does not do?

Not at the moment. I'll play with it further.

comment:28 follow-up: Changed 6 years ago by egourgoulhon

PS: Another relevant project is K3D, which was presented at Sage Days 74:

https://wiki.sagemath.org/K3D-tools

comment:29 in reply to: ↑ 28 Changed 6 years ago by paulmasson

Replying to egourgoulhon:

PS: Another relevant project is K3D, which was presented at Sage Days 74:

https://wiki.sagemath.org/K3D-tools

It's good to know about this project, but I'm aiming for something more general and not focused just on notebooks. Matplotlib lets you create and save image files for 2d plots. I'd like something equivalent for 3d plots, which for Three.js is a self-contained HTML file.

comment:30 in reply to: ↑ 27 Changed 6 years ago by paulmasson

Replying to egourgoulhon:

Replying to kcrisman:

  • this would be a very good enhancement, though I guess it would have to then depend on the backend.

Maybe not: looking at the source code, in line 155 of src/sage/plot/plot3d/threejs.py, there is

addLabel( 'x=' + ( xMid ).toFixed(d).toString(), xMid, b[1].y+offset, a[2]*b[0].z );

It seems easy to replace 'x=' by another string.

Since this is done in a browswer, and they all understand Unicode now, there won't be issues with missing fonts like in matplotlib. This is an easy customization.

comment:31 in reply to: ↑ 26 Changed 6 years ago by paulmasson

Replying to kcrisman:

Can you think of anything it does not do? I seem to recall that there were things jsmol did with shading etc. that three.js couldn't, when this was discussed ~ 5 years ago.

I would find it difficult to believe that current Three.js can't match anything in Jmol. The transparency issue mentioned above can likely be handled with the alphaMap property that appeared in Three.js about the same time as the comment, and if not that way then by a custom shader which isn't difficult to implement.

comment:32 in reply to: ↑ 25 ; follow-up: Changed 6 years ago by paulmasson

Replying to egourgoulhon:

Wow! I've just tried it and it's very nice! Thank you very much for this enhancement!

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

Thanks for trying it out! I'm trying to decide between reading the show() options of the plots fed into threejs() or just have a separate dictionary, but should probably do the latter. That would include options for setting axes labels, setting digits after the decimal in labels, turning off axes labels altogether, turning off the bounding box and setting an explicit aspect ratio in all directions. Let me know what other options would be interesting.

There is already automatic scaling in the z-direction to avoid the tall thin box that appears in the current version on the cell server (and SMC), but that can easily be overridden. I can also easily add face colors to support surfaces with color maps. Lights for the scene are meant to be customizable as well and can stand some additional improvement.

The initial camera position is not yet completely adaptive to the possible variety of input plots, so ideas on that would be welcome.

comment:33 Changed 6 years ago by mkoeppe

  • Cc yzh added

comment:34 follow-up: Changed 6 years ago by novoselt

We most definitely want this in, so please keep working!!!

Note that we have optional threejs package (SageMathCell is using it) - if we have a functioning viewer we can easily make is standard as it is small and very fast to install.

Interface-wise let's stick to show for the actual showing of stuff and plot/plot3d for producing plots that can be further manipulated. threejs command if it exists at all should do something like set up a low-level threejs scene that can be tweaked as advanced users please. Presumably, it is possible to just pass some HTML/JavaScript code that will be inserted into generated files?

comment:35 follow-up: Changed 6 years ago by novoselt

File-wise: perhaps instead of raw string it is possible to have template_three.js file that will be read/cached by the Python function? Perhaps with template_three.js.html to further wrap it into a standalone page. One of the advantages for development - syntax highlighting of this long template.

comment:36 follow-up: Changed 6 years ago by novoselt

So it seems that at the moment opacity is ignored and line thickness is not taken into account - is it possible to implement these?

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

comment:37 follow-up: Changed 6 years ago by novoselt

What about embedding the resulting plots into web-page, i.e. how easy it is for someone-not-really-knowing-anything to take a plot from Sage and have it running on their web-page? (No using SageMathCell, just having the three.js stuff properly saved/inserted somewhere.)

comment:38 in reply to: ↑ 37 Changed 6 years ago by paulmasson

Replying to novoselt:

What about embedding the resulting plots into web-page, i.e. how easy it is for someone-not-really-knowing-anything to take a plot from Sage and have it running on their web-page? (No using SageMathCell, just having the three.js stuff properly saved/inserted somewhere.)

The entire aim is to have a self-contained HTML file with everything, all geometric data and the JavaScript to process it. Then it's simply a matter of adding an iframe to a web page and setting its src attribute to the HTML file. Easy peasy!

comment:39 in reply to: ↑ 36 ; follow-ups: Changed 6 years ago by paulmasson

Replying to novoselt:

So it seems that at the moment opacity is ignored and line thickness is not taken into account - is it possible to implement these?

Opacity should be simple. Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

Thought I had this figured out. For a plot with the origin at the center it behaves as you expect, but not when the center is displaced. Will definitely work on this.

comment:40 in reply to: ↑ 39 ; follow-up: Changed 6 years ago by novoselt

Replying to paulmasson:

Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

I consider this to be a major flaw, no matter whose fault it is - on the one hand it is nice to use different thicknesses and on the other - single pixel width typically looks bad when used with projectors... Are there any sensible workarounds? Like text seems to be scaled and then scaled back - can it be done with lines too?

comment:41 in reply to: ↑ 34 Changed 6 years ago by paulmasson

Replying to novoselt:

We most definitely want this in, so please keep working!!! Interface-wise let's stick to show for the actual showing of stuff and plot/plot3d for producing plots that can be further manipulated. threejs command if it exists at all should do something like set up a low-level threejs scene that can be tweaked as advanced users please.

A scene in Sage is a collection of Graphics3d objects. Three.js, like Jmol/JSmol, is merely a way to render the scene. I don't see the use for command to set up a separate Three.js scene from scratch.

Until this Three.js viewer is integrated into the Rich Output backend, I don't have a way to display the scene without a command other than show, which will default to Jmol/JSmol. That can change later once an initial version is working acceptably. I've spent enough time reading the Rich Output backend to know where to make those changes, and at that point show can then be set to default to a Three.js viewer if desired. For now I need the separate command!

Presumably, it is possible to just pass some HTML/JavaScript code that will be inserted into generated files?

See the next comment for this.

comment:42 Changed 6 years ago by novoselt

No problem with having a separate command now, but what I meant is - let's not introduce and then deprecate it in Sage, so using rich output should be done on this ticket as well.

comment:43 in reply to: ↑ 35 Changed 6 years ago by paulmasson

Replying to novoselt:

File-wise: perhaps instead of raw string it is possible to have template_three.js file that will be read/cached by the Python function? Perhaps with template_three.js.html to further wrap it into a standalone page. One of the advantages for development - syntax highlighting of this long template.

I can only think of two ways to communicate data from Python to JavaScript at the command line: either with the Python-formatted string I'm using, or by appending data to a URL as a query string and processing it from there. This second way leads to monstrous URLs that can be bookmarked but not saved as stand-alone files. With what I'm doing all of the data is embedded in the HTML file: saving the file saves the entire scene.

On the server you have other options for processing data, but running something like that at the command line seems to me to be massive overkill. If I'm missing something in your comment, please let me know a better way to transfer the data to the browser.

Also, the raw string is not proper JavaScript because all curly brackets are doubled to become literals during Python formatting. Saving that as a separate file would produce parsing errors.

comment:44 follow-up: Changed 6 years ago by novoselt

Sorry I was not clear again: what I meant is take your current template, save it as js file in the same Sage source folder, then in Python code open this file, read it as string, and do the same processing as what you do now. So it will not affect the end result for the user, just, IMHO, makes it easier to maintain this template inside of Sage.

comment:45 in reply to: ↑ 40 Changed 6 years ago by paulmasson

Replying to novoselt:

Replying to paulmasson:

Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

I consider this to be a major flaw, no matter whose fault it is - on the one hand it is nice to use different thicknesses and on the other - single pixel width typically looks bad when used with projectors... Are there any sensible workarounds? Like text seems to be scaled and then scaled back - can it be done with lines too?

Text is done on an HTML canvas, so that's completely different from WebGL lines. There are workarounds, but I'd have to think about it more and then would process lines the same way for all browsers.

comment:46 in reply to: ↑ 44 Changed 6 years ago by paulmasson

Replying to novoselt:

Sorry I was not clear again: what I meant is take your current template, save it as js file in the same Sage source folder, then in Python code open this file, read it as string, and do the same processing as what you do now. So it will not affect the end result for the user, just, IMHO, makes it easier to maintain this template inside of Sage.

When the Python part of this file is integrated with the Rich Output backend, then all that will be left in this file will be the template, so sure. But again, this is not proper JavaScript because of the curly bracket doubling.

comment:47 follow-up: Changed 6 years ago by novoselt

Ah, and that's because you want to use .format(..) with this string. An alternative is to use %s substitution if it makes sense or just regular string replacement, but if you prefer doubling so be it since you are doing the actual work ;-)

comment:48 Changed 6 years ago by kcrisman

Just wanted to say this sounds like a great discussion, keep it up! It would be great to have something reliable. Also you may want to check out how this looks/works in the Jupyter notebook as well as sagenb.

comment:49 Changed 6 years ago by kcrisman

(I assume it will be fine, just mentioning.)

comment:50 Changed 6 years ago by paulmasson

  • Status changed from needs_review to needs_work

comment:51 Changed 6 years ago by git

  • Commit changed from 7cedf75db69671cfb0a201435e5148d1041f0f05 to f4b1c03a27ec37fc9d6470879f4364300bcda1be

Branch pushed to git repo; I updated commit sha1. New commits:

f4b1c03Move template to separate file

comment:52 in reply to: ↑ 47 Changed 6 years ago by paulmasson

Replying to novoselt:

Ah, and that's because you want to use .format(..) with this string. An alternative is to use %s substitution if it makes sense or just regular string replacement, but if you prefer doubling so be it since you are doing the actual work ;-)

Should have thought of string replacement! Template now separate. Much preferable to have proper JavaScript in it.

comment:53 Changed 6 years ago by git

  • Commit changed from f4b1c03a27ec37fc9d6470879f4364300bcda1be to 4f7075dfcd3cfcca01e3e6f14cc4e5153748c67c

Branch pushed to git repo; I updated commit sha1. New commits:

df51bdeUpdate initial camera position
4f7075dFix label scaling

comment:54 in reply to: ↑ 39 Changed 6 years ago by paulmasson

Replying to paulmasson:

Replying to novoselt:

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

Thought I had this figured out. For a plot with the origin at the center it behaves as you expect, but not when the center is displaced. Will definitely work on this.

This should be fixed now.

comment:55 in reply to: ↑ 32 Changed 6 years ago by egourgoulhon

Replying to paulmasson:

That would include options for setting axes labels, setting digits after the decimal in labels, turning off axes labels altogether, turning off the bounding box and setting an explicit aspect ratio in all directions. Let me know what other options would be interesting.

It would be nice to have some (customizable) ticks along the bounding box, but this is probably somewhat tricky and should be the topic of another ticket.

Besides, what about text3d?

At the moment, when running threejs from the Jupyter notebook, the 3d plot is rendered in another tab of the browser. Could it be embedded in the notebook instead? (maybe this is related to the Rich Output issue).

Thanks again for your work on this!

comment:56 Changed 6 years ago by git

  • Commit changed from 4f7075dfcd3cfcca01e3e6f14cc4e5153748c67c to 62e7a8baaf9f4f193fbf1ff8b3406f2aa634d68d

Branch pushed to git repo; I updated commit sha1. New commits:

62e7a8bIntegrate with IPython command line

comment:57 Changed 6 years ago by paulmasson

  • Description modified (diff)

Basic integration with the backend for the IPython command line has been added, and the usage has changed accordingly. This was important to understand how options are passed through the backend. Integration with notebooks will be added after basic display options are in place.

Note: See TracTickets for help on using tickets.