Opened 10 years ago

Last modified 8 months ago

#12402 closed enhancement

Make a three.js backend for 3d plotting — at Version 72

Reported by: jason Owned by: jason, was
Priority: major Milestone: sage-7.5
Component: graphics Keywords:
Cc: kini, novoselt, mbejger, gagern, egourgoulhon, paulmasson, yzh, mkoeppe Merged in:
Authors: Paul Masson Reviewers:
Report Upstream: N/A Work issues:
Branch: u/paulmasson/12402 (Commits, GitHub, GitLab) Commit: 46dcb962b6888cdc83b4262f58c4596db70b5f2d
Dependencies: Stopgaps:

Status badges

Change History (73)

comment:1 Changed 10 years ago by kini

  • Cc kini added

comment:2 Changed 10 years ago by novoselt

  • Cc novoselt added

comment:3 Changed 10 years ago by was

  • Description modified (diff)

comment:4 Changed 9 years ago by kcrisman

Jason, William, can you put whatever code exists on here since public worksheets are disabled? We don't want this just in the cloud.sagemath :-)

Changed 9 years ago by jason

comment:6 follow-up: Changed 9 years ago by kcrisman

Just as an update, this is now apparently working in cloud.sagemath.

comment:7 in reply to: ↑ 6 Changed 9 years ago by was

Replying to kcrisman:

Just as an update, this is now apparently working in cloud.sagemath.

It sort of works, but it isn't included by default. It will be soon though, so that anybody can at least test out some minimal functionality. I'll post when that happens.

comment:8 Changed 9 years ago by jdemeyer

  • Milestone changed from sage-5.11 to sage-5.12

comment:9 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.1 to sage-6.2

comment:10 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.2 to sage-6.3

comment:11 Changed 8 years ago by mbejger

  • Cc mbejger added

comment:12 follow-up: Changed 8 years ago by jason

Just FYI, there are now three different threejs backends: SMC, the threejs command in the sage cell, and the pythreejs-based backend that works in the sage cell too.

comment:13 Changed 8 years ago by novoselt

Jason - what is the status of the last one? Will it be pushed to get "standard" or there is still work to be done there and with your switch nobody is available to steer it?

The first two were definitely lacking features, in particular "things that don't scale with zoom" plus some transparency issues.

comment:14 follow-up: Changed 8 years ago by jason

The transparency issues are not fixed---those are inherent in how three.js draws things, and stem from limitations of webgl (there are experiements overcoming them in webgl, which might be interesting to look at, though; the author of mathbox is looking at them, for example).

I think that anything "pushed to standard" should be supported by SMC and the sage cell. Right now, the pythreejs solution requires an communication framework that is/emulates the ipython comm/widget framework. Something like that would need to be implemented for SMC to get it to work there. I think that would be a great project in itself, though---there is starting to be some good development of interactive things for the IPython notebook, and it would be great to use those things from SMC too.

comment:15 in reply to: ↑ 14 Changed 8 years ago by kcrisman

I think that anything "pushed to standard" should be supported by SMC and the sage cell. Right now, the pythreejs solution requires an communication framework that is/emulates the ipython comm/widget framework. Something like that would need to be implemented for SMC to get it to work there. I think that would be a great project in itself, though---there is starting to be some good development of interactive things for the IPython notebook, and it would be great to use those things from SMC too.

But I thought SMC only uses the js plotting - does it also do Jmols too? (You'll notice I still haven't had time to properly try out SMC, and won't for some time).


See also #16004 as a related ticket I also haven't had enough time to review properly.

comment:16 in reply to: ↑ 12 ; follow-up: Changed 8 years ago by mbejger

Replying to jason:

Just FYI, there are now three different threejs backends: SMC, the threejs command in the sage cell, and the pythreejs-based backend that works in the sage cell too.

Thanks :-) I am interested in enhancing the capabilities of SageManifolds? (#14865), which is still under development, so I guess the most natural thing would be to somehow use threejs locally. Probably this question was asked already many times, but what is the current status of threejs in sage notebook (local ipython notebook)?

comment:17 in reply to: ↑ 16 Changed 8 years ago by kcrisman

Thanks :-) I am interested in enhancing the capabilities of SageManifolds? (#14865), which is still under development, so I guess the most natural thing would be to somehow use threejs locally. Probably this question was asked already many times, but what is the current status of threejs in sage notebook (local ipython notebook)?

The Sagenb is not really at all an Ipython notebook, though it may have inspired earlier versions thereof. Anyway, the current status is that you can get it to work - see the examples above - but no one has put time into this. Partly because the sagenb is in maintenance mode currently (though it would be nice to have it more actively developed) but also because of the slight backwards functionality loss involved as you see in previous comments.

comment:18 Changed 8 years ago by gagern

  • Cc gagern added

comment:19 Changed 8 years ago by vbraun_spam

  • Milestone changed from sage-6.3 to sage-6.4

comment:20 Changed 8 years ago by egourgoulhon

  • Cc egourgoulhon added

comment:21 Changed 6 years ago by paulmasson

  • Cc paulmasson added

comment:22 Changed 6 years ago by paulmasson

  • Branch set to u/paulmasson/12402

comment:23 Changed 6 years ago by paulmasson

  • Commit set to 7cedf75db69671cfb0a201435e5148d1041f0f05
  • Description modified (diff)
  • Milestone changed from sage-6.4 to sage-7.4
  • Status changed from new to needs_review

Here's some basic Three.js support for Graphics3d objects. It is modeled on the corresponding function in SageMathCell: simply use threejs() to wrap any combination of points, lines and surfaces. This will save an HTML file with the JavaScript needed to render the scene and open it in a browser.

There are a great many features that can be added in the future, but I first would like to see how much interest there is in this. There have been problems surfacing with Jmol (#20978 for example) and I certainly think it would be preferable to have a native viewer that can be updated more quickly. The HTML format is also more portable than Jmol files, since it only requires a modern browser to run.

This version relies on an external CDN to retrieve the Three.js library, since it does not currently download to the hard drive during the build process. I'll need help on figuring out how to change that at some point, since I'm sure people will want to be able to view saved files offline.


Last 10 new commits:

3d1cd2aHandle nested lists
efb3d0aHandle nested lists
f70ac3dAdd line plots
55f12e6Add point plots
b4c8414Adjust centering and camera
374809aRebase to move point bounds off points
9a42352Add auto z-aspect
d560683Minor rebase
aa322adAdd axis labels
7cedf75Check for equal bounds

comment:24 Changed 6 years ago by paulmasson

  • Authors set to Paul Masson

comment:25 follow-ups: Changed 6 years ago by egourgoulhon

Wow! I've just tried it and it's very nice! Thank you very much for this enhancement!

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

comment:26 in reply to: ↑ 25 ; follow-ups: Changed 6 years ago by kcrisman

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

See e.g. here - this would be a very good enhancement, though I guess it would have to then depend on the backend.

Can you think of anything it does not do? I seem to recall that there were things jsmol did with shading etc. that three.js couldn't, when this was discussed ~ 5 years ago.

comment:27 in reply to: ↑ 26 ; follow-up: Changed 6 years ago by egourgoulhon

Replying to kcrisman:

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

See e.g. here

Yes, set_axes_labels is the (not very satisfactory) workaround I wrote to deal with this when using Jmol ;-)

  • this would be a very good enhancement, though I guess it would have to then depend on the backend.

Maybe not: looking at the source code, in line 155 of src/sage/plot/plot3d/threejs.py, there is

addLabel( 'x=' + ( xMid ).toFixed(d).toString(), xMid, b[1].y+offset, a[2]*b[0].z );

It seems easy to replace 'x=' by another string.

Can you think of anything it does not do?

Not at the moment. I'll play with it further.

comment:28 follow-up: Changed 6 years ago by egourgoulhon

PS: Another relevant project is K3D, which was presented at Sage Days 74:

https://wiki.sagemath.org/K3D-tools

comment:29 in reply to: ↑ 28 Changed 6 years ago by paulmasson

Replying to egourgoulhon:

PS: Another relevant project is K3D, which was presented at Sage Days 74:

https://wiki.sagemath.org/K3D-tools

It's good to know about this project, but I'm aiming for something more general and not focused just on notebooks. Matplotlib lets you create and save image files for 2d plots. I'd like something equivalent for 3d plots, which for Three.js is a self-contained HTML file.

comment:30 in reply to: ↑ 27 Changed 6 years ago by paulmasson

Replying to egourgoulhon:

Replying to kcrisman:

  • this would be a very good enhancement, though I guess it would have to then depend on the backend.

Maybe not: looking at the source code, in line 155 of src/sage/plot/plot3d/threejs.py, there is

addLabel( 'x=' + ( xMid ).toFixed(d).toString(), xMid, b[1].y+offset, a[2]*b[0].z );

It seems easy to replace 'x=' by another string.

Since this is done in a browswer, and they all understand Unicode now, there won't be issues with missing fonts like in matplotlib. This is an easy customization.

comment:31 in reply to: ↑ 26 Changed 6 years ago by paulmasson

Replying to kcrisman:

Can you think of anything it does not do? I seem to recall that there were things jsmol did with shading etc. that three.js couldn't, when this was discussed ~ 5 years ago.

I would find it difficult to believe that current Three.js can't match anything in Jmol. The transparency issue mentioned above can likely be handled with the alphaMap property that appeared in Three.js about the same time as the comment, and if not that way then by a custom shader which isn't difficult to implement.

comment:32 in reply to: ↑ 25 ; follow-up: Changed 6 years ago by paulmasson

Replying to egourgoulhon:

Wow! I've just tried it and it's very nice! Thank you very much for this enhancement!

A question: is it possible to customize the axes labels (e.g. via the keyword argument axes_labels, as in 2D plots)?

Thanks for trying it out! I'm trying to decide between reading the show() options of the plots fed into threejs() or just have a separate dictionary, but should probably do the latter. That would include options for setting axes labels, setting digits after the decimal in labels, turning off axes labels altogether, turning off the bounding box and setting an explicit aspect ratio in all directions. Let me know what other options would be interesting.

There is already automatic scaling in the z-direction to avoid the tall thin box that appears in the current version on the cell server (and SMC), but that can easily be overridden. I can also easily add face colors to support surfaces with color maps. Lights for the scene are meant to be customizable as well and can stand some additional improvement.

The initial camera position is not yet completely adaptive to the possible variety of input plots, so ideas on that would be welcome.

comment:33 Changed 6 years ago by mkoeppe

  • Cc yzh added

comment:34 follow-up: Changed 6 years ago by novoselt

We most definitely want this in, so please keep working!!!

Note that we have optional threejs package (SageMathCell is using it) - if we have a functioning viewer we can easily make is standard as it is small and very fast to install.

Interface-wise let's stick to show for the actual showing of stuff and plot/plot3d for producing plots that can be further manipulated. threejs command if it exists at all should do something like set up a low-level threejs scene that can be tweaked as advanced users please. Presumably, it is possible to just pass some HTML/JavaScript code that will be inserted into generated files?

comment:35 follow-up: Changed 6 years ago by novoselt

File-wise: perhaps instead of raw string it is possible to have template_three.js file that will be read/cached by the Python function? Perhaps with template_three.js.html to further wrap it into a standalone page. One of the advantages for development - syntax highlighting of this long template.

comment:36 follow-up: Changed 6 years ago by novoselt

So it seems that at the moment opacity is ignored and line thickness is not taken into account - is it possible to implement these?

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

comment:37 follow-up: Changed 6 years ago by novoselt

What about embedding the resulting plots into web-page, i.e. how easy it is for someone-not-really-knowing-anything to take a plot from Sage and have it running on their web-page? (No using SageMathCell, just having the three.js stuff properly saved/inserted somewhere.)

comment:38 in reply to: ↑ 37 Changed 6 years ago by paulmasson

Replying to novoselt:

What about embedding the resulting plots into web-page, i.e. how easy it is for someone-not-really-knowing-anything to take a plot from Sage and have it running on their web-page? (No using SageMathCell, just having the three.js stuff properly saved/inserted somewhere.)

The entire aim is to have a self-contained HTML file with everything, all geometric data and the JavaScript to process it. Then it's simply a matter of adding an iframe to a web page and setting its src attribute to the HTML file. Easy peasy!

comment:39 in reply to: ↑ 36 ; follow-ups: Changed 6 years ago by paulmasson

Replying to novoselt:

So it seems that at the moment opacity is ignored and line thickness is not taken into account - is it possible to implement these?

Opacity should be simple. Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

Thought I had this figured out. For a plot with the origin at the center it behaves as you expect, but not when the center is displaced. Will definitely work on this.

comment:40 in reply to: ↑ 39 ; follow-up: Changed 6 years ago by novoselt

Replying to paulmasson:

Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

I consider this to be a major flaw, no matter whose fault it is - on the one hand it is nice to use different thicknesses and on the other - single pixel width typically looks bad when used with projectors... Are there any sensible workarounds? Like text seems to be scaled and then scaled back - can it be done with lines too?

comment:41 in reply to: ↑ 34 Changed 6 years ago by paulmasson

Replying to novoselt:

We most definitely want this in, so please keep working!!! Interface-wise let's stick to show for the actual showing of stuff and plot/plot3d for producing plots that can be further manipulated. threejs command if it exists at all should do something like set up a low-level threejs scene that can be tweaked as advanced users please.

A scene in Sage is a collection of Graphics3d objects. Three.js, like Jmol/JSmol, is merely a way to render the scene. I don't see the use for command to set up a separate Three.js scene from scratch.

Until this Three.js viewer is integrated into the Rich Output backend, I don't have a way to display the scene without a command other than show, which will default to Jmol/JSmol. That can change later once an initial version is working acceptably. I've spent enough time reading the Rich Output backend to know where to make those changes, and at that point show can then be set to default to a Three.js viewer if desired. For now I need the separate command!

Presumably, it is possible to just pass some HTML/JavaScript code that will be inserted into generated files?

See the next comment for this.

comment:42 Changed 6 years ago by novoselt

No problem with having a separate command now, but what I meant is - let's not introduce and then deprecate it in Sage, so using rich output should be done on this ticket as well.

comment:43 in reply to: ↑ 35 Changed 6 years ago by paulmasson

Replying to novoselt:

File-wise: perhaps instead of raw string it is possible to have template_three.js file that will be read/cached by the Python function? Perhaps with template_three.js.html to further wrap it into a standalone page. One of the advantages for development - syntax highlighting of this long template.

I can only think of two ways to communicate data from Python to JavaScript at the command line: either with the Python-formatted string I'm using, or by appending data to a URL as a query string and processing it from there. This second way leads to monstrous URLs that can be bookmarked but not saved as stand-alone files. With what I'm doing all of the data is embedded in the HTML file: saving the file saves the entire scene.

On the server you have other options for processing data, but running something like that at the command line seems to me to be massive overkill. If I'm missing something in your comment, please let me know a better way to transfer the data to the browser.

Also, the raw string is not proper JavaScript because all curly brackets are doubled to become literals during Python formatting. Saving that as a separate file would produce parsing errors.

comment:44 follow-up: Changed 6 years ago by novoselt

Sorry I was not clear again: what I meant is take your current template, save it as js file in the same Sage source folder, then in Python code open this file, read it as string, and do the same processing as what you do now. So it will not affect the end result for the user, just, IMHO, makes it easier to maintain this template inside of Sage.

comment:45 in reply to: ↑ 40 Changed 6 years ago by paulmasson

Replying to novoselt:

Replying to paulmasson:

Line thickness can be included, but will not be consistent across operating systems because of a limitation in Microsoft's implementation of WebGL.

I consider this to be a major flaw, no matter whose fault it is - on the one hand it is nice to use different thicknesses and on the other - single pixel width typically looks bad when used with projectors... Are there any sensible workarounds? Like text seems to be scaled and then scaled back - can it be done with lines too?

Text is done on an HTML canvas, so that's completely different from WebGL lines. There are workarounds, but I'd have to think about it more and then would process lines the same way for all browsers.

comment:46 in reply to: ↑ 44 Changed 6 years ago by paulmasson

Replying to novoselt:

Sorry I was not clear again: what I meant is take your current template, save it as js file in the same Sage source folder, then in Python code open this file, read it as string, and do the same processing as what you do now. So it will not affect the end result for the user, just, IMHO, makes it easier to maintain this template inside of Sage.

When the Python part of this file is integrated with the Rich Output backend, then all that will be left in this file will be the template, so sure. But again, this is not proper JavaScript because of the curly bracket doubling.

comment:47 follow-up: Changed 6 years ago by novoselt

Ah, and that's because you want to use .format(..) with this string. An alternative is to use %s substitution if it makes sense or just regular string replacement, but if you prefer doubling so be it since you are doing the actual work ;-)

comment:48 Changed 6 years ago by kcrisman

Just wanted to say this sounds like a great discussion, keep it up! It would be great to have something reliable. Also you may want to check out how this looks/works in the Jupyter notebook as well as sagenb.

comment:49 Changed 6 years ago by kcrisman

(I assume it will be fine, just mentioning.)

comment:50 Changed 6 years ago by paulmasson

  • Status changed from needs_review to needs_work

comment:51 Changed 6 years ago by git

  • Commit changed from 7cedf75db69671cfb0a201435e5148d1041f0f05 to f4b1c03a27ec37fc9d6470879f4364300bcda1be

Branch pushed to git repo; I updated commit sha1. New commits:

f4b1c03Move template to separate file

comment:52 in reply to: ↑ 47 Changed 6 years ago by paulmasson

Replying to novoselt:

Ah, and that's because you want to use .format(..) with this string. An alternative is to use %s substitution if it makes sense or just regular string replacement, but if you prefer doubling so be it since you are doing the actual work ;-)

Should have thought of string replacement! Template now separate. Much preferable to have proper JavaScript in it.

comment:53 Changed 6 years ago by git

  • Commit changed from f4b1c03a27ec37fc9d6470879f4364300bcda1be to 4f7075dfcd3cfcca01e3e6f14cc4e5153748c67c

Branch pushed to git repo; I updated commit sha1. New commits:

df51bdeUpdate initial camera position
4f7075dFix label scaling

comment:54 in reply to: ↑ 39 Changed 6 years ago by paulmasson

Replying to paulmasson:

Replying to novoselt:

Also when I scale the image text grows/shrinks and then "becomes normal again" - is it possible to always keep it the same size throughout the transition?

Thought I had this figured out. For a plot with the origin at the center it behaves as you expect, but not when the center is displaced. Will definitely work on this.

This should be fixed now.

comment:55 in reply to: ↑ 32 ; follow-up: Changed 6 years ago by egourgoulhon

Replying to paulmasson:

That would include options for setting axes labels, setting digits after the decimal in labels, turning off axes labels altogether, turning off the bounding box and setting an explicit aspect ratio in all directions. Let me know what other options would be interesting.

It would be nice to have some (customizable) ticks along the bounding box, but this is probably somewhat tricky and should be the topic of another ticket.

Besides, what about text3d?

At the moment, when running threejs from the Jupyter notebook, the 3d plot is rendered in another tab of the browser. Could it be embedded in the notebook instead? (maybe this is related to the Rich Output issue).

Thanks again for your work on this!

comment:56 Changed 6 years ago by git

  • Commit changed from 4f7075dfcd3cfcca01e3e6f14cc4e5153748c67c to 62e7a8baaf9f4f193fbf1ff8b3406f2aa634d68d

Branch pushed to git repo; I updated commit sha1. New commits:

62e7a8bIntegrate with IPython command line

comment:57 Changed 6 years ago by paulmasson

  • Description modified (diff)

Basic integration with the backend for the IPython command line has been added, and the usage has changed accordingly. This was important to understand how options are passed through the backend. Integration with notebooks will be added after basic display options are in place.

comment:58 in reply to: ↑ 55 Changed 6 years ago by paulmasson

Replying to egourgoulhon:

Replying to paulmasson:

That would include options for setting axes labels, setting digits after the decimal in labels, turning off axes labels altogether, turning off the bounding box and setting an explicit aspect ratio in all directions. Let me know what other options would be interesting.

It would be nice to have some (customizable) ticks along the bounding box, but this is probably somewhat tricky and should be the topic of another ticket.

Indeed!

Besides, what about text3d?

This object does not have a json_repr and that would be overkill anyway. I can easily add optional text labels like those on the frame, but would need to think about how to automatically extract text3d from other Graphics3d objects.

At the moment, when running threejs from the Jupyter notebook, the 3d plot is rendered in another tab of the browser. Could it be embedded in the notebook instead? (maybe this is related to the Rich Output issue).

It can certainly be embedded, and that is precisely dependent on integration with the Rich Output backend. To be done.

Thanks again for your work on this!

comment:59 Changed 6 years ago by git

  • Commit changed from 62e7a8baaf9f4f193fbf1ff8b3406f2aa634d68d to 4f2b9f1c5677582a8e4ba9bf1a74d0b896ad98c3

Branch pushed to git repo; I updated commit sha1. New commits:

a93418aImplement basic options
4f2b9f1Update temporary threejs() command

comment:60 follow-up: Changed 6 years ago by paulmasson

  • Status changed from needs_work to needs_review

I've now implement several basic display options:

frame (bool) display bounding box or not

aspect_ratio (list) individual aspect multipliers for each axis

axes_labels (list) labels to display. If set to false, all labels are removed.

decimals (integer) decimal points in labels

opacity (float) overall opacity for surfaces. Currently set for the entire scene.

I've also updated the temporary threejs() command so that it has the new options. This command will be maintained until integration with both notebook backends is complete.

comment:61 in reply to: ↑ 60 Changed 6 years ago by egourgoulhon

I've tried the display options: they work well, except for aspect_ratio:

threejs(icosahedron(), aspect_ratio=[0.5, 1., 0.8])

returns

TypeError: 0.500000000000000 is not JSON serializable

comment:62 follow-up: Changed 6 years ago by egourgoulhon

May I ask about the status of this? As I understand, except for the aspect_ratio option and the Rich Output backend, this is almost ready. It would be so nice if this could be shipped with Sage 7.5!

comment:63 follow-up: Changed 6 years ago by was

I somehow just looked at this for a few minutes -- take or leave these comments. Feel free to ignore them...

The first thing I see (in https://git.sagemath.org/sage.git/tree/src/sage/plot/plot3d/threejs_template.html?id=67930e843d29652f143c67329aaf32be4ab120a8) is:

<script src=http://rawgit.com/mrdoob/three.js/r80/build/three.js></script>
<script src=http://rawgit.com/mrdoob/three.js/r80/examples/js/controls/OrbitControls.js></script>

A concern is that this won't work offline -- 3d graphics will just be completely broken without an external internet connection. Also, this is making 3d graphics display in Sage depend on "some random site" (rawgit.com) which says on their page: "The catch: this is a free service, so there are no uptime or support guarantees."...

This provides absolutely no Jupyter notebook integration at all as far as I can tell. It's just creating a single standalone local website for the image and popping that up in the users browser. If you do something more serious that does try to integrate with Jupyter, you might find it is a lot of work to get Chrome with more than about 10+ 3d plots on the same page.. That was a big problem when I implemented WebGL based 3d graphics for SageMathCloud?, which involved quite a bit of work (involving great care with the renderer, swapping things out with static png's etc.). When you're just trying to get a demo working you don't worry about such scale. However, if you're teaching, you can easily have many plots in one notebook. Anyway, this is one reason that the SMC Javascript is longer and more complicated.

Another question is how much of 3d plot rendering have you actually implemented? For example, I recently added arrowheads to lines to SMC, since I had missed those and you expect them when plotting 3d vector fields. I don't see any code in threejs_template.html for that.

I guess you just only support WebGL, even though three.js can support canvas too.... Maybe that's fine these days, but again supporting both is something I put a lot of work into.

The SMC implementation:

comment:64 Changed 6 years ago by kcrisman

Great comments, especially the offline bit.

comment:65 in reply to: ↑ 63 Changed 6 years ago by paulmasson

Replying to was:

A concern is that this won't work offline -- 3d graphics will just be completely broken without an external internet connection.

This is easily fixed by adding a second pair of script tags pointing to a local source. Since Three.js does not currently download during the build process, adding this feature will have to change that and is more appropriate for another ticket.

Also, this is making 3d graphics display in Sage depend on "some random site" (rawgit.com) which says on their page: "The catch: this is a free service, so there are no uptime or support guarantees."...

Rawgit.com is hardly some random site for GitHub people. That being said it's irrelevant where the code is hosted as long as it is available somewhere. The intent is to make the interactive image as portable as a PNG, so it can't depend entirely on one's local installation: an external CDN link is vital.

This provides absolutely no Jupyter notebook integration at all as far as I can tell. It's just creating a single standalone local website for the image and popping that up in the users browser.

The process here of creating 3D images exactly parallels that of 2D plots: a file is generated, saved to disc and displayed in the best possible viewer as determined by the rich output backend. It's not a local website, it's a file that can be saved and used elsewhere as desired, exactly as for static 2D plots.

If you do something more serious that does try to integrate with Jupyter, you might find it is a lot of work to get Chrome with more than about 10+ 3d plots on the same page.. That was a big problem when I implemented WebGL based 3d graphics for SageMathCloud?, which involved quite a bit of work (involving great care with the renderer, swapping things out with static png's etc.).

I'm well aware of the limitations of multiple WebGL contexts in a single web page. The real way to handle this is using scissoring on a single renderer: I wrote an example for the Three.js repository to do just that. Since this approach is not immediately modular, the fallback is to manage iframe content as it scrolls into view. While I was figuring out scissoring, a coding friend of mine explored that very subject to good effect, so we have an available resource for dealing with this issue as well. It's also a topic for another ticket, since it would require modifying notebook behavior. I can't control all of that in the process of creating stand-alone files. The motivation here is to bring Three.js to the command line interface first.

Another question is how much of 3d plot rendering have you actually implemented? For example, I recently added arrowheads to lines to SMC, since I had missed those and you expect them when plotting 3d vector fields. I don't see any code in threejs_template.html for that.

The code will currently render anything that returns a json_repr. This includes the arrow graphics primitive, which appears just fine, but not for example text3d. 3D vector fields don't have arrow heads in Sage 7.4 anyway, so that is something that can be added on another ticket.

I guess you just only support WebGL, even though three.js can support canvas too.... Maybe that's fine these days, but again supporting both is something I put a lot of work into.

This is also easily fixed by testing rendering capabilities, but I'd rather look forward for the initial version.

Last edited 6 years ago by paulmasson (previous) (diff)

comment:66 in reply to: ↑ 62 ; follow-up: Changed 6 years ago by paulmasson

Replying to egourgoulhon:

May I ask about the status of this? As I understand, except for the aspect_ratio option and the Rich Output backend, this is almost ready. It would be so nice if this could be shipped with Sage 7.5!

Eric, there's a bit more work to be done:

1) All numeric input parameters need to be made JSON serializable. This is easy to do.

2) The Three.js Geometry that I used for convenience needs to be replaced with BufferGeometry. This will improve use of available memory and avoid possible browser issues, as well as prepare for stability in notebooks. This is straightforward.

3) The code used for line objects needs to be completely rewritten. There is a hardware driver limitation on Windows machines that prevents rendering of line thickness in WebGL that is not going away soon, and I'm still thinking about the best way around this. Andrey certainly won't approve this ticket without line thickness rendering decently across hardware platforms, and I agree with him on that.

4) Notebook integration needs to be added and tested. Since I am not familiar with the details of notebooks, this may not be straightforward without input from those more familiar. Adding a few Three.js files to a notebook page should be easy, but more than that may require modifying how the notebook dynamically handles iframe content. That might also be better dealt with on a separate ticket.

I've been distracted in the last couple weeks with trying to transpile Sage libraries with Emscripten for use directly in the browser, but since that's more work that I thought it would be I can set it aside for awhile and get back to this!

Last edited 6 years ago by paulmasson (previous) (diff)

comment:67 Changed 6 years ago by paulmasson

  • Milestone changed from sage-7.4 to sage-7.5
  • Status changed from needs_review to needs_work

comment:68 in reply to: ↑ 66 ; follow-up: Changed 6 years ago by egourgoulhon

Replying to paulmasson:

Thanks for your answers.

3) The code used for line objects needs to be completely rewritten. There is a hardware driver limitation on Windows machines that prevents rendering of line thickness in WebGL that is not going away soon, and I'm still thinking about the best way around this. Andrey certainly won't approve this ticket without line thickness rendering decently across hardware platforms, and I agree with him on that.

I would be on a different tone here: as long as three.js is not Sage's default 3D viewer, I see no reason to defer its use for Linux and MacOS users, simply because Windows is limited. When decision is taken to make three.js the default viewer, it's of course a different matter. So IMHO the Windows line thickness issue could be postponed to another ticket.

I've been distracted in the last couple weeks with trying to transpile Sage libraries with Emscripten for use directly in the browser, but since that's more work that I thought it would be I can set it aside for awhile and get back to this!

Thanks!

comment:69 in reply to: ↑ 68 Changed 6 years ago by slelievre

Replying to egourgoulhon:

Replying to paulmasson:

Thanks for your answers.

3) The code used for line objects needs to be completely rewritten. There is a hardware driver limitation on Windows machines that prevents rendering of line thickness in WebGL that is not going away soon, and I'm still thinking about the best way around this. Andrey certainly won't approve this ticket without line thickness rendering decently across hardware platforms, and I agree with him on that.

I would be on a different tone here: as long as three.js is not Sage's default 3D viewer, I see no reason to defer its use for Linux and MacOS users, simply because Windows is limited. When decision is taken to make three.js the default viewer, it's of course a different matter. So IMHO the Windows line thickness issue could be postponed to another ticket.

True, Windows line thickness could be its own ticket, and making three.js the default viewer could be another ticket depending on it.

I've been distracted in the last couple weeks with trying to transpile Sage libraries with Emscripten for use directly in the browser, but since that's more work that I thought it would be I can set it aside for awhile and get back to this!

Thanks!

Excellent. A recent Ask Sage question about tick marks on the frame of 3d plots points to the timeliness of this.

comment:70 Changed 6 years ago by git

  • Commit changed from 4f2b9f1c5677582a8e4ba9bf1a74d0b896ad98c3 to 0fea7503fbb586beda18b4f83ba54a6ed7143638

Branch pushed to git repo; I updated commit sha1. New commits:

1dd3f80Fix points background
0fea750Fix JSON serialization error

comment:71 Changed 6 years ago by git

  • Commit changed from 0fea7503fbb586beda18b4f83ba54a6ed7143638 to 46dcb962b6888cdc83b4262f58c4596db70b5f2d

Branch pushed to git repo; I updated commit sha1. New commits:

53d6decMinor edit
cabe3eeAdd basic Jupyter notebook support
0e19bfcAdd basic Sage notebook support
46dcb96Improvements to labels

comment:72 Changed 6 years ago by paulmasson

  • Description modified (diff)
  • Status changed from needs_work to needs_review

Turns out the BufferGeometry isn't necessary for getting this working in notebooks. I've added some basic integration for both notebook types parallel to how JSmol is embedded in each.

Caveat: WebGL is limited to 16 active contexts at a time, so while all the data is in the notebooks the figures may not be active until the cell is reevaluated. I have a basic idea of how to fix this in the future, but it would require a modification to the notebooks themselves which would be a separate ticket.

I haven't worked on the line thickness issue yet.

Note: See TracTickets for help on using tickets.