#21267 closed enhancement (fixed)
Port SageNB widgets/interact to Jupyter
Reported by:  jdemeyer  Owned by:  

Priority:  major  Milestone:  sage8.0 
Component:  notebook  Keywords:  
Cc:  charpent  Merged in:  
Authors:  Jeroen Demeyer  Reviewers:  Emmanuel Charpentier 
Report Upstream:  N/A  Work issues:  
Branch:  1046695 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  Stopgaps: 
Description (last modified by )
Support old SageNB widgets in Jupyter notebooks. This is done by writing wrapper functions with the old SageNB names returning ipywidgets
widgets. That is, we essentially do
def input_box(...): return ipywidgets.Text(...)
The current implementation should be 100% compatible with ipywidgets
, just extending it with Sage functionality.
We also try to support the old SageNB @interact
functionality. Many examples of SageNB interacts can be found on https://wiki.sagemath.org/interact and we try to support as much as possible in Jupyter.
These are the changes in functionality that I know of:
slider(0,10)
returns a slider which runs over the integers from 0 to 10 (in SageNB, this ran over floats from 0.0 to 10.0).
range_slider()
with rational numbers is not implemented: https://github.com/ipython/ipywidgets/issues/760 (now implemented in https://github.com/jupyterwidgets/ipywidgets/pull/1356)
 The output is updated only once when a button is clicked multiple times: https://github.com/ipython/ipywidgets/issues/763 (the needed hook to implement this is done in https://github.com/jupyterwidgets/ipywidgets/pull/1259)
 Labels do not support HTML, only plain text and LaTeX: https://github.com/ipython/ipywidgets/issues/817
 The visual presentation of widgets can be different.
Tarballs:
Change History (64)
comment:1 followup: 2 Changed 6 years ago by
comment:2 followup: 11 Changed 6 years ago by
Replying to vbraun:
Are there any API conflicts?
At least, the API is mostly compatible. I don't know any case where SageNB and Jupyter have significantly different behaviour. For the things that SageNB supports but Jupyter does not, either drop support for that feature or work with upstream to implement the feature in Jupyter. I don't strive to achieve 100% compatibility, I'm happy if it mostly works.
In some cases, the visual presentation might be different (Jupyter using a dropdown box while SageNB uses something like radio buttons). I don't care about that.
One potential issue is that SageNB supports all kinds of numbers (Integer
, RealNumber
, Rational
, ...) for sliders while Jupyter only supports int
and float
.
comment:3 Changed 6 years ago by
Branch:  → u/jdemeyer/port_sagenb_widgets_interact_to_jupyter 

comment:4 Changed 6 years ago by
Commit:  → f9e723b65a8ed6f87fdee4471a22f71b68330755 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
f9e723b  Port SageNB interacts to Jupyter

comment:6 Changed 6 years ago by
Description:  modified (diff) 

comment:7 Changed 6 years ago by
Replying to kcrisman:
WOW how did I not know about this ticket!
I'll let you know when I need testers.
comment:8 Changed 6 years ago by
Cc:  charpent added 

I posted a temporary workaround for people wanting interactive graphs in Jupyter notebooks using the Sage kernel.
comment:9 Changed 6 years ago by
For the record, I consider this branch mostly featurecomplete. It still lacks documentation and tests and the required changes to the ipywidgets
package.
I have not decided what to do with the upstream situation. There are 3 options:
 add a whole lot of patches to
build/pkgs/ipywidgets
 package the upstream
master
branch  wait for
ipywidgets6
to be released
comment:10 Changed 6 years ago by
Commit:  f9e723b65a8ed6f87fdee4471a22f71b68330755 → 8704fc0f4000250e4b4c9474c60c6877c56c10b7 

Branch pushed to git repo; I updated commit sha1. New commits:
8704fc0  Fix selector with labels

comment:11 followup: 12 Changed 6 years ago by
Replying to jdemeyer:
... I don't strive to achieve 100% compatibility, I'm happy if it mostly works.
I say that it is quite important to work towards such a goal for all interfaces: SageNB, Jupyter, SageMathCloud, SageMathCell. It is highly annoying to develop a nice interact and then discover that it does not work with another interface. Perhaps SageNB can be left behind now, but I hope that other 3 can converge to a single documented standard...
comment:12 Changed 6 years ago by
Replying to novoselt:
I say that it is quite important to work towards such a goal for all interfaces: SageNB, Jupyter, SageMathCloud, SageMathCell. It is highly annoying to develop a nice interact and then discover that it does not work with another interface. Perhaps SageNB can be left behind now, but I hope that other 3 can converge to a single documented standard...
I agree that we should work towards such a goal but some things are not so easy at the moment. In particular, this patch is reusing the existing widgets from ipywidgets
, it doesn't add completely new widgets (it doesn't change any Javascript). For example, ipywidgets
doesn't have a slider with rational numbers so we cannot have that. Luckily, this can be emulated by a slider taking an arbitrary list of values. But this trick does not work for a range_slider
, so a range_slider
with rational numbers is not supported here.
comment:13 Changed 6 years ago by
IMHO, sliders behave strangely in SageNB:
@interact def f(x=(1, 10)): print(x, parent(x))
This gives a slider with floatingpoint values between 1 and 10. In ipywidgets
, this would become a slider with integer values between 1 and 10. I think that the ipywidgets
convention makes more sense here: the type of the variable should correspond to the type in the interact abbrevation (x=(1, 10)
gives integers while x=(1.0, 10.0)
gives floats).
On top of this, this gives Python float
s while in a Sage context, an element of RR
would be more appropriate.
comment:14 Changed 6 years ago by
Commit:  8704fc0f4000250e4b4c9474c60c6877c56c10b7 → 6b0dd4c15dd7656a18242918a808c3345f2806ce 

Branch pushed to git repo; I updated commit sha1. New commits:
6b0dd4c  Allow unicode labels

comment:15 Changed 6 years ago by
Commit:  6b0dd4c15dd7656a18242918a808c3345f2806ce → 66decb63c93693b6ebd494ce9789b321107a47a9 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
66decb6  Allow unicode labels

comment:16 Changed 6 years ago by
Dependencies:  #21256 → #21552 

comment:17 Changed 6 years ago by
Commit:  66decb63c93693b6ebd494ce9789b321107a47a9 → 41e476d675819a8a122827c0ebb39040ace82ddd 

comment:18 Changed 6 years ago by
Description:  modified (diff) 

comment:19 Changed 6 years ago by
Commit:  41e476d675819a8a122827c0ebb39040ace82ddd → d8e04b196be4099539574e4272bc8b32ddcf6979 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
d8e04b1  Port SageNB interacts to Jupyter

comment:20 Changed 6 years ago by
Description:  modified (diff) 

comment:21 Changed 6 years ago by
Description:  modified (diff) 

comment:22 Changed 6 years ago by
Dependencies:  #21552 

Description:  modified (diff) 
Milestone:  sage7.4 → sage7.6 
comment:23 Changed 6 years ago by
Description:  modified (diff) 

comment:24 Changed 6 years ago by
Dependencies:  → #22119 

comment:25 Changed 6 years ago by
Commit:  d8e04b196be4099539574e4272bc8b32ddcf6979 → 19cde93272ad976d2c82f2df84fecee9eb611a39 

comment:26 Changed 6 years ago by
Dependencies:  #22119 → #22119, #22125 

comment:27 Changed 6 years ago by
Commit:  19cde93272ad976d2c82f2df84fecee9eb611a39 → 147b9808f35f6624052c8641a4f61a66ad657801 

comment:28 Changed 6 years ago by
Commit:  147b9808f35f6624052c8641a4f61a66ad657801 → 4957bbc0c71436104685619e550208c8c80b2eaa 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
4957bbc  Port SageNB interacts to Jupyter

comment:29 Changed 6 years ago by
Commit:  4957bbc0c71436104685619e550208c8c80b2eaa → f443f89c308d159ce863e37857813c47826911a6 

comment:30 followup: 31 Changed 6 years ago by
Description:  modified (diff) 

comment:31 Changed 6 years ago by
Replying to jdemeyer:
Is this upgrade ready for review ? Or do you need any form of help before formal review ?
comment:32 Changed 6 years ago by
Essentially yes, it's ready.
I didn't formally set the ticket to needs_review because I'd like to wait until the stable release of ipywidgets
is released.
comment:33 Changed 6 years ago by
Description:  modified (diff) 

comment:34 Changed 6 years ago by
Description:  modified (diff) 

comment:35 Changed 6 years ago by
Commit:  f443f89c308d159ce863e37857813c47826911a6 → 7ce92bbcbfdabfc8f9750ebdce079ddfd99df420 

comment:36 Changed 6 years ago by
Commit:  7ce92bbcbfdabfc8f9750ebdce079ddfd99df420 → 64ee490ff787c44d357ec456471196f065f584cd 

comment:37 Changed 6 years ago by
Dependencies:  #22119, #22125 

comment:38 Changed 6 years ago by
Description:  modified (diff) 

comment:39 Changed 6 years ago by
Commit:  64ee490ff787c44d357ec456471196f065f584cd → 6e6013aea58a92f2b242995b899a083b3d55a183 

comment:40 Changed 6 years ago by
Description:  modified (diff) 

comment:41 Changed 6 years ago by
Description:  modified (diff) 

comment:42 Changed 6 years ago by
Commit:  6e6013aea58a92f2b242995b899a083b3d55a183 → 053ad423c67f9d23ff527cd3a1307e89d277dadd 

comment:43 Changed 6 years ago by
Milestone:  sage7.6 → sage8.0 

comment:44 Changed 6 years ago by
Commit:  053ad423c67f9d23ff527cd3a1307e89d277dadd → 30955d2ec72338cc7b13c17922389bb58c8f278c 

comment:45 Changed 6 years ago by
Status:  new → needs_review 

comment:46 Changed 6 years ago by
Reviewers:  → Emmanuel Charpentier 

Status:  needs_review → positive_review 
starting with a freshly fetched 7.6.beta4, make distclean && make ptestlong
passes with one wellknownfailure :
 sage t long src/sage/homology/simplicial_complex.py # 1 doctest failed 
which passes when ran standalone.
A couple warnings are repeated in widgetsnbextension2.0.0rc1.log
and ipywidgets6.0.0.rc2.log
:
/usr/local/sageexp/local/lib/python2.7/sitepackages/setuptools/dist.py:285: UserWarning: Normalizing '2.0.0.rc1' to '2.0.0rc1' warning: manifest_maker: standard file 'c' not found /usr/local/sageexp/local/lib/python2.7/sitepackages/setuptools/dist.py:285: UserWarning: Normalizing '6.0.0.rc2' to '6.0.0rc2'
which look like innocuous booboos.
==> positive_review
For the record : congratulations for this intricate, lengthy and much expected patch. Thank you very much !
I'd recommend expedited inclusion of this update, which should go a long way making the Jupyter notebook as functional as the old one.
I plan to merge this one in my current working branch (which includes #20523 an its dependencies) and give it a couple "reallife" functional tests. But unless the results turns out to be non or barelyfunctional (which would surprise me), further enhancements should go towards a new ticket.
comment:47 followups: 48 55 Changed 6 years ago by
Status:  positive_review → needs_work 

Damn !
from the PREP tutorial "Sage Interact Quickstart" :
sage: @interact sage: def _(f=input_box(x^2,width=20), ....: color=color_selector(widget='colorpicker', label=""), ....: axes=True, ....: fill=True, ....: zoom=range_slider(3,3,default=(3,3))): ....: show(plot(f,(x,zoom[0], zoom[1]), color=color, axes=axes,fill=fill, figsize=3))
(I added the figsize=3
)
gives a static fgure, with no controls. I see that in the terminal windows from where I launched the notebook :
[IPKernelApp] WARNING  The installed widget Javascript is the wrong version. It must satisfy the semver range ~2.1.0.
One hidden dependency ? Something to document (in plain English...) ?
Or something more serious ?
BTW : this works well in the old Sage notebook (same binary, ran from another terminal...).
comment:48 followup: 49 Changed 6 years ago by
Replying to charpent:
Damn !
from the PREP tutorial "Sage Interact Quickstart" :
sage: @interact sage: def _(f=input_box(x^2,width=20), ....: color=color_selector(widget='colorpicker', label=""), ....: axes=True, ....: fill=True, ....: zoom=range_slider(3,3,default=(3,3))): ....: show(plot(f,(x,zoom[0], zoom[1]), color=color, axes=axes,fill=fill, figsize=3))(I added the
figsize=3
)gives a static fgure, with no controls. I see that in the terminal windows from where I launched the notebook :
[IPKernelApp] WARNING  The installed widget Javascript is the wrong version. It must satisfy the semver range ~2.1.0.One hidden dependency ? Something to document (in plain English...) ?
Your example works for me. My guess is that it's something like a browser cache which might interfere. Can you try clearing the cache and try again?
comment:49 followups: 50 51 Changed 6 years ago by
Replying to jdemeyer:
Replying to charpent:
[ Snip ...]
Your example works for me. My guess is that it's something like a browser cache which might
interfere. Can you try clearing the cache and try again?
Done. Same result.
I might have another problem : when I type "SAGE_BROWSER=chromium sage n jupyter", Sage still opens a Firefox sheet.
Logging out of this sheet, manually opening Chromium, and pasting the address given in the controlling terminal window allows to open the same sheet in Chrome, with the same result.
Suggestions ?
comment:50 Changed 6 years ago by
One more data point : same results (incl. Firefox starting with SAGE_BROWSER="chromium") after rebooting ths whole hawg...
I'm at loss.
comment:51 followup: 52 Changed 6 years ago by
Replying to charpent:
Suggestions ?
This is most likely not Sagerelated, so please report this issue upstream to ipywidgets: https://github.com/ipython/ipywidgets/issues
comment:52 Changed 6 years ago by
Replying to jdemeyer:
Replying to charpent:
Suggestions ?
This is most likely not Sagerelated, so please report this issue upstream to ipywidgets: https://github.com/ipython/ipywidgets/issues
This is now ipywidget's issue 1611.
comment:53 Changed 6 years ago by
Description:  modified (diff) 

comment:54 Changed 6 years ago by
Commit:  30955d2ec72338cc7b13c17922389bb58c8f278c → 10466954904447ad0df31bc559acb36121e4b40c 

Branch pushed to git repo; I updated commit sha1. New commits:
1046695  Fix exceptions in interacts

comment:55 Changed 6 years ago by
Status:  needs_work → needs_review 

Replying to charpent:
[IPKernelApp] WARNING  The installed widget Javascript is the wrong version. It must satisfy the semver range ~2.1.0.
Do you happen to have another installation of ipywidgets
/widgetsnbextension
which might interfere?
I'm setting this back to needs_review because I cannot reproduce your problem. I hope that other people will try it, so we can see if it works for them.
comment:56 Changed 6 years ago by
Status:  needs_review → positive_review 

I tried your patch on two different installations (one physical machine, one pristine VM) where it succeeded. Coming back to my initial test case, I took the speps to remove entierely ipywidgets an widgetsnbextension from the systemwide installs. Theren your patch worked.
I conclude that, somehow, the systemwide installation(s) of Python extensions take precedence over a sagespecific installation. hence a couple questions :
 Is that expected or is that a bug ?
 If a bug, does it deserve a ticket ?
 If expected (not a bug), shouldn't this behavior be documented ?
In any case, This lifts my objections ==> positive_review
.
And kudos !
comment:57 Changed 6 years ago by
Upstream is aware of issues with conflicting installations. In fact that's why I made the suggestion in 55 to check other installations.
comment:58 Changed 6 years ago by
Branch:  u/jdemeyer/port_sagenb_widgets_interact_to_jupyter → 10466954904447ad0df31bc559acb36121e4b40c 

Resolution:  → fixed 
Status:  positive_review → closed 
comment:59 followup: 60 Changed 6 years ago by
Commit:  10466954904447ad0df31bc559acb36121e4b40c 

@ Jeroen: quick question, I notice you have added two patches for ipywidgets
. Are those upstreamed? If so can you give some references?
comment:60 followup: 61 Changed 6 years ago by
Replying to fbissey:
@ Jeroen: quick question, I notice you have added two patches for
ipywidgets
.
Actually, there are 3 patches. Two of them (get_interact_value_output.patch
and output_exception.patch
) should be in https://pypi.python.org/pypi/ipywidgets/6.0.0.rc3
The other one (widget_repr.patch
) is being discussed at https://github.com/ipython/ipywidgets/pull/1031 but upstream isn't very happy about it. However, this one is really only relevant for doctesting, not for runtime functionality.
comment:61 Changed 6 years ago by
Replying to jdemeyer:
Replying to fbissey:
@ Jeroen: quick question, I notice you have added two patches for
ipywidgets
.Actually, there are 3 patches. Two of them (
get_interact_value_output.patch
andoutput_exception.patch
) should be in https://pypi.python.org/pypi/ipywidgets/6.0.0.rc3
OK, good!
The other one (
widget_repr.patch
) is being discussed at https://github.com/ipython/ipywidgets/pull/1031 but upstream isn't very happy about it. However, this one is really only relevant for doctesting, not for runtime functionality.
OK, I missed that one completely, I suppose that is why I have
File "/usr/lib64/python2.7/sitepackages/sage/repl/ipython_kernel/widgets.py", line 349, in sage.repl.ipython_kernel.widgets.Grid.__init__ Failed example: w Expected: Grid(value=[[0, 1], [4, 5]], description=u'2x2 matrix', children=(Label(value=u'2x2 matrix'), VBox(children=(EvalText(value=u'0'), EvalText(value=u'4'))), VBox(children=(EvalText(value=u'1'), EvalText(value=u'5'))))) Got: <sage.repl.ipython_kernel.widgets.Grid object at 0x7f8d65145f90>
And indeed it doesn't seem to have an impact on functionality.
comment:63 Changed 6 years ago by
Description:  modified (diff) 

I updated the description to reflect recent changes in upstream ipywidgets.
comment:64 Changed 6 years ago by
Thanks for the pointers, I will certainly follow up on this (but maybe not right now).
Are there any API conflicts? And if yes, how are we dealing with that? Separate
@jupyterinteract
or what? A global switch?