Opened 3 years ago

Last modified 6 weeks ago

#24593 needs_work enhancement

Replace Thebe by ThebeLab for live documentation support

Reported by: nthiery Owned by:
Priority: major Milestone: sage-9.3
Component: documentation Keywords: notebook
Cc: jdemeyer, tmonteil, embray, vdelecroix, jhpalmieri, klee Merged in:
Authors: Nicolas M. Thiéry, Jeroen Demeyer Reviewers:
Report Upstream: N/A Work issues:
Branch: u/jdemeyer/upgrade_from_thebe_to_thebelab_for_live_documentation_support (Commits) Commit: c9910413af226d145632e8a1bb7b7e0dd4a67720
Dependencies: Stopgaps:

Description (last modified by jdemeyer)

Thebe is a javascript library to turn static HTML pages into live documents where code cells can be edited and executed. Thebe was been introduced in #20690 as a replacement for the live documentation feature of sagenb. It covers the use case of browsing the documentation through the Jupyter notebook; in this case, Thebe connected to that notebook for computations.

However this model was not sustainable:

  • Shipping Thebe is additional burden for Sage's maintainers and packagers
  • Thebe is a fork of the Jupyter code base and thus not maintainable.
  • In practice, Thebe got out of sync and soon became incompatible with the Jupyter we shipped. So the feature was lost.

Meanwhile, Thebe has been reimplemented as ThebeLab, a thin layer on top of JupyterLab?, making it much more sustainable. In addition, an online computation backend is now available thanks to http://mybinder.org and Sage's docker container.

In this ticket, we switch gear, and focus on the use case of browsing the documentation from anywhere (within Jupyter, local static documentation, Sage's online documentation), with the assumption of an internet connection.

Implementation:

  • When compiling the documentation, Sphinx adds a small header to the produced static html pages.
  • If a page contains a sage code cell, an Activate button is added.
  • Upon activation, and only then, the thebelab javascript library together with its dependencies in the jupyterlab javascript library are fetched from the web.
  • Thebe is configured to use Sage's docker container on MyBinder? as computation backend.

Bonus:

  • Get rid of Thebe's package.
  • A few bells and whistles of the new Thebe implementation (status field, ...)
  • #21680 can be closed

The feature has been tested for a couple months on More SageMath Tutorials. It's also configured by default for any Sage package that uses sage-package; see e.g. this page.

We believe that dropping the use case of offline browsing while having a notebook running (which is broken anyway) is largely compensated by the much easier maintenance and the availability of the feature for all deployments of Sage's documentation.

thebelab tarball: https://registry.npmjs.org/thebelab/-/thebelab-0.2.1.tgz

Attachments (3)

thebelab-0.2.1.tar.bz2 (207.1 KB) - added by nthiery 2 years ago.
test-thebe.ipynb (3.2 KB) - added by nthiery 2 years ago.
thebelab-0.2.1.2.tgz (233.6 KB) - added by nthiery 2 years ago.

Download all attachments as: .zip

Change History (103)

comment:1 Changed 3 years ago by nthiery

  • Description modified (diff)

comment:2 Changed 3 years ago by nthiery

  • Description modified (diff)

comment:3 Changed 3 years ago by nthiery

  • Branch set to u/nthiery/upgrade_from_thebe_to_thebelab_for_live_documentation_support

comment:4 Changed 3 years ago by nthiery

  • Commit set to b4497c130de334ebe2f651e41c5301b754770406
  • Description modified (diff)

New commits:

b4497c124593: Upgrade from Thebe to ThebeLab for live documentation support

comment:5 Changed 3 years ago by nthiery

I am currently building the whole documentation on my machine to test it locally.

comment:6 Changed 3 years ago by nthiery

  • Cc jdemeyer tmonteil embray added

comment:7 Changed 3 years ago by nthiery

Works here, either when opening the html doc files directly in the browser as file:..., or by browsing the documentation through the Help menu of the Jupyter notebook.

Hence needs review.

comment:8 Changed 3 years ago by nthiery

  • Cc vdelecroix added
  • Status changed from new to needs_review

comment:9 Changed 3 years ago by nthiery

I had a look at the patchbot reports; one is gree. Some of the patchbot fails on compiling giac: most likely unrelated. For another the startup time is affected; again, it seems likely to be unrelated.

comment:10 Changed 3 years ago by nthiery

See sage-devel for the giac failure: apparently due to gcc that needs rebuilding (https://trac.sagemath.org/ticket/24599) .

comment:11 Changed 3 years ago by tmonteil

I am not sure to completely understand how the whole stuff work, here are some questions:

  • The thebelab javascript is provided by a CDN (namely unpkg.com), and since the version is hardcoded in the link, if there is an upgrade of thebelab, there will be an update within Sage source code as well, so why not providing the javascript locally (by packaging thebelab), and avoid this remote intermediary ?
  • The work of forking jupyterlab to extract the thebe-like part was done already by thebelab. Why not using that interesting part locally and use the running jupyter instance instead of mybinder ?
  • How could a user running the doc from her Sage installation be sufficiently warned that, while a Sage kernel is currently running on her machine, the code will however be executed remotely ?
  • The local documentation depends on the version of Sage (e.g. existence of some method, option, ...), hence there must be a corresponding kernel on mybinder:
    • How is the local Sage version transmitted to mybinder so that the correct kernel is launched remotely ?
    • This require a frequent updates on the mybinder machines (one at each release). Who takes the responsibility of long-term maintenance of remote kernels ?
  • What is the point of having this into Sagemath source code if Sagemath can not run it, as it requires remote servers to work anyway ? Why not instead maintaining a livedoc.sagemath.org service ? This would be more consistent and understandable from a user perspective.
  • Is it possible to disable this feature when building Sage documentation (as we did for the plot directive) ?
  • [this one is sarcastic] shall we consider having sage --notebook open a webpage redirecting to some cocalc or any similar centralized, bandwidth consuming, earth warming, privacy harming, open-source alternative ?

comment:12 Changed 3 years ago by tmonteil

  • Status changed from needs_review to needs_info

comment:13 Changed 3 years ago by vdelecroix

I don't understand the following

Thebe is configured to use Sage's docker container on !MyBinder as
computation backend.

This means that you need to use MyBinder in order to have live documentation? What is the point of having Sage on my computer then? To my mind, having a live documentation needing internet and a web service is not worth it.

comment:14 Changed 3 years ago by vdelecroix

To balance comment:13: I think that using MyBinder would be good for let say livedoc.sagemath.org as Thierry suggested in comment:11.

comment:15 follow-up: Changed 3 years ago by novoselt

What exactly are the differences between Thebe and SageMathCell? In particular:

  • Do multiple users share the same running instance of the container?
  • How many concurrent users can be supported?
  • How long can a session run, i.e. if there are several linked cells, how long can one wait after executing the first one before executing the last one?
  • What are CPU/memory/disk usage limits?

Regarding others comments: I also do not see the point in having my local documentation (which is build from the source on my machine, perhaps my own branch) to use for computations remote servers. Having a link (perhaps glued on the screen as the current Activate button) to some web server with live cells would be much less confusing in terms of accessing remote resources and different versions.

comment:16 Changed 3 years ago by nthiery

Hi everyone,

Thanks for the thoughtfull comments. I am done with teaching for the week, and can finally come back to this.

I realized I forgot to state the obvious: yes, when possible, using a locally running jupyter would be preferable for computations for all the aforementionned reasons (use of resources, privacy, consistency with the locally installed kernel, ...).

But this requires some additional work:

  1. packaging thebelab
  2. packaging jupyterlab
  3. implementing the logic: "am I being served by a Jupyter server? If yes, connect there; otherwise offer binder as backup".
  4. making sure that the jupyterlab client code is compatible with the Jupyter server we are running

I therefore aimed for a strategy in two stages:

  • Implement something simple for now, put it in production, and assess how robust this is
  • Add support for using a local Jupyter server when available

Additional motivations:

  • the use of the local jupyter by our previous Thebe setup was broken anyway, This ticket tackles the most urgent: reinstating the feature in most use cases.
  • we might as well wait for Sage to upgrade from Jupyter to JupyterLab? before implementing the second step, to save on 2. and 4.

What do you think?

Now to more specific points:

  • The thebelab javascript is provided by a CDN (namely unpkg.com), and since the version is hardcoded in the link, if there is an upgrade of thebelab, there will be an update within Sage source code as well, so why not providing the javascript locally (by packaging thebelab), and avoid this remote intermediary ?

The link is actually using semantic versioning (that's the ^ in thebelab@^0.1.0). So it will fetch the latest version of thebelab whose API is meant to be backward compatible.

  • The work of forking jupyterlab to extract the thebe-like part was done already by thebelab.

Just for clarification: it's not a fork. thebelab is using the vanilla jupyterlab as a library.

  • How could a user running the doc from her Sage installation be sufficiently warned that, while a Sage kernel is currently running on her machine, the code will however be executed remotely ?

Yes, that's an important point. The tooltip explains this. Do you have specific suggestions? We could e.g. easily change the Activate button title or the messages that appear when it get pressed.

The live documentation, is mostly meant for casually toying with small variations of the existing examples; this sounds unlikely to leak private information. That being said, yes, the user should be well aware of what's happening in case she would want to play harder.

  • This require a frequent updates on the mybinder machines (one at each release). Who takes the responsibility of long-term maintenance of remote kernels ?

The bulk of the work is to update the docker container, which have many other use cases anyway. This is mostly automated. Erik and Julian are working on fully automatizing this, toward including this in the release process.

Beyond this, there remains on trivial item: updating the label in:

https://github.com/sagemath/sage-binder-env/blob/master/Dockerfile

I am happy to volunteer on that for the next couple years :-)

alternative the livedoc.sagemath.org approach

Our users browse the Sage documentation in many ways: on doc.sagemath.org, through the notebook's Help menu, or browsing directly the local html pages, or a copy of them on their department page. With this ticket (and admittedly assuming internet), the user can make the documentation live right where he is. Forcing her/him to navigate to a parallel set of documentation pages is only making things more complicated. And breaks the flow of reading.

Besides this would also forces us to setup and maintain a new service.

  • Is it possible to disable this feature when building Sage documentation (as we did for the plot directive) ?

It should be doable, but I don't necessarily see the point. Except for the 10 lines of javascript/css to setup the button, nothing happens unless the user explicitly presses the activate button: no connection to binder, no thebelab download, no network activity of any form.

About thebelab and the Sage cell server

That's a real question, as there is much overlap between the two projects. We actually discussed this with Jason Grout last week.

ThebeLab?, like Jupyter, is language agnostic and targets a much much wider community than the Sage cell server. It's therefore likely to get more traction in the long run, in terms of users, developers, and computing resources. It can also use any Jupyter server (a local jupyter, a jupyterhub, binder, ...) that the user has credentials for.

The Sage cell server on the other hand is much more advanced when it comes to performance. For example, when used with binder, a full new jupyter ends up being started each time, rather than just spawning a new kernel, or even forking an existing one.

I would love to see a convergence between the two projects, with the cool features and ideas of the Sage cell server being ported over to ThebeLab? (or presumably down the stack into Jupyter/JupyterHub?). Kind of what happened with the Sage notebook and Jupyter.

Cheers,

Nicolas

Last edited 3 years ago by nthiery (previous) (diff)

comment:17 in reply to: ↑ 15 Changed 3 years ago by nthiery

Replying to novoselt:

What exactly are the differences between Thebe and SageMathCell? In particular:

  • Do multiple users share the same running instance of the container?

No

  • How many concurrent users can be supported?

I don't know for sure. Since the project has been revamped a couple months ago, Binder seems to scale relatively well. There are several ongoing efforts as well to add more deployements (like this one on the "EU cloud") though they are not federated yet.

  • How long can a session run, i.e. if there are several linked cells, how long can one wait after executing the first one before executing the last one?
  • What are CPU/memory/disk usage limits?

I would need to check on the binder docs.

I remember beeing really annoyed by Sage's live documentation that would loose its history in the middle of my lectures. But I have changed workflow and haven't used ThebeLab? enough to have a feeling on whether this could become annoying for a typical usage.

comment:18 Changed 3 years ago by schilly

Hi, I see this here for the first time. Since I'm the one maintaining the public sage documentation files, will this an impact on it? Since that last "thebe" change, I always have to apply a patch to Sage in order to make it work (basically, to disable the javascript). https://github.com/sagemath/documentation/blob/gh-pages/thebe.patch

Is there also an intention to make this work for the doc.sagemath.org pages?

comment:19 Changed 3 years ago by nthiery

Hi Harald,

Hi, I see this here for the first time. Since I'm the one maintaining the public sage documentation files, will this an impact on it? Since that last "thebe" change, I always have to apply a patch to Sage in order to make it work (basically, to disable the javascript). https://github.com/sagemath/documentation/blob/gh-pages/thebe.patch

Is there also an intention to make this work for the doc.sagemath.org pages?

Thanks for investigating this. Yes, having this work on doc.sagemath.org is one of the big point.

The new implementation consists of a couple lines of javascript and css, which just assume standard internet connection to further fetch thebelab's javascript from a cdn and connect to binder. This worked without special action on ReadTheDoc?. See e.g.:

http://sage-package.readthedocs.io/en/latest/sage_package/sphinx-demo.html

Do you foresee anything specific in the doc.sagemath.org setup that could break that? If useful, I can provide with a tiny tarball with one example page and the js and css to be temporarily deployed in a corner of docs.sagemath.org for testing.

Cheers,

Nicolas

comment:20 follow-up: Changed 3 years ago by schilly

Do you foresee anything specific in the doc.sagemath.org setup ...

No, there is nothing weird going on. It's just static files hosted on the "fastly" CDN.

OTOH, I also don't remember what the problem is right now. Probably that it didn't load at all or something like that. That's why I disabled it.

comment:21 in reply to: ↑ 20 Changed 3 years ago by nthiery

No, there is nothing weird going on. It's just static files hosted on the "fastly" CDN.

Good. Then this should "just work".

OTOH, I also don't remember what the problem is right now. Probably that it didn't load at all or something like that. That's why I disabled it.

Thebe is currently (i.e. before this ticket) configured in the Sage documentation to connect to a local notebook. So the feature could never have worked on doc.sagemath.org anyway. It could be that, in addition, the detection logic was faulty and causing problem.

comment:22 follow-up: Changed 3 years ago by nthiery

  • Status changed from needs_info to needs_review

I believe I provided tentative answers to the questions that were asked, so I am setting this back to needs review. Mostly as a ping.

Any opinions? In particular on the two stages approach?

comment:23 follow-up: Changed 3 years ago by novoselt

How could a user running the doc from her Sage installation be sufficiently warned that, while a Sage kernel is currently running on her machine, the code will however be executed remotely ?

Yes, that's an important point. The tooltip explains this. Do you have specific suggestions? We could e.g. easily change the Activate button title or the messages that appear when it get pressed.

I think it should be much more clear that a remote server will be used and, perhaps, a different version of Sage. How about some dialog window with this information that appears when "Activate" is pressed? Perhaps with a checkbox "Don't show this again". Relying on users reading tooltips is unreliable...

comment:24 in reply to: ↑ 23 ; follow-up: Changed 3 years ago by nthiery

Replying to novoselt:

I think it should be much more clear that a remote server will be used and, perhaps, a different version of Sage.

How about some dialog window with this information that appears when "Activate" is pressed? Perhaps with a checkbox "Don't show this again". Relying on users reading tooltips is unreliable...

Dialog windows are a bit of a bother for users. I had a look at how you handle the same issue in SageMatCell?. I like the "Help" and "About" links. A further improvement would be to have those links along the run button rather than at the end of the output: first to see them before actually pressing button, and also to save on vertical space.

This would need to be done upstream in Thebe though; I'll post a feature request there. It's also somewhat more complicated than for the SageMathCell since there can be many possible configurations, and the message or page linked to should reflect that.

In the mean time, one easy thing to do would be to adapt the status messages that are displayed in the status field that replaces the Activate button. Currently it shows:

  • building
  • launching
  • ready

We could make it into something like:

  • Setting up Thebe
  • Connecting to mybinder.org
  • Connected to mybinder.og

where mybinder.org would be a clickable link.

What do you think? Suggestions for better wording?

comment:25 in reply to: ↑ 24 Changed 3 years ago by novoselt

Replying to nthiery:

Replying to novoselt:

I think it should be much more clear that a remote server will be used and, perhaps, a different version of Sage.

How about some dialog window with this information that appears when "Activate" is pressed? Perhaps with a checkbox "Don't show this again". Relying on users reading tooltips is unreliable...

Dialog windows are a bit of a bother for users. I had a look at how you handle the same issue in SageMatCell?. I like the "Help" and "About" links. A further improvement would be to have those links along the run button rather than at the end of the output: first to see them before actually pressing button, and also to save on vertical space.

This is not at all the same issue: SageMathCell handles cells embedded in random web pages without any relation to local installation. If a user opens some website, it is only natural that the content is using the internet. But when I build a local copy of Sage, perhaps with my own modifications, and local files from this copy use a remote server with a different version of Sage - that's not at all obvious.

I do agree that dialog windows are annoying, yet we had them for TOS when it was necessary and they would appear only once. I did not find this annoying at all and for showing an important and non-obvious piece of information that's a great solution, I think.

Regarding possible improvement for SageMathCell - I've added a link to some existing ones. Presumably the output location was used as the one which is always visible (at least when things are working) while input can be completely hidden including the button.

comment:26 in reply to: ↑ 22 ; follow-up: Changed 3 years ago by vdelecroix

Replying to nthiery:

I believe I provided tentative answers to the questions that were asked, so I am setting this back to needs review. Mostly as a ping.

Any opinions? In particular on the two stages approach?

For the time being, we could simply disable thebe (since you say that it is broken). Having in Sage an even more broken feature (= let users depend on internet and mybinder) is not an improvement. To my mind, the proposal in this ticket is for online documentation only (and a possible good proposal for that purpose). However, it has nothing to do inside the standard Sage.

The current branch is not like a huge amount of code and refactoring. So I don't see any emergency in including it at this stage. People who are interested in testing/using it can simply apply the branch as it is done for many experimental features.

comment:27 in reply to: ↑ 26 Changed 3 years ago by nthiery

Replying to vdelecroix:

For the time being, we could simply disable thebe (since you say that it is broken). Having in Sage an even more broken feature (= let users depend on internet and mybinder) is not an improvement.

Yes it is an improvement for the local documentation! It goes from non functional to functional -- admitedly with caveats -- when the user has internet.

Think in particular about the case where the reader is not browsing through a Jupyter server; e.g. because she opened the files directly with the browser or is on a local copy on a department page; then using the local Sage setup, as desirable as it would be, would be really hard: how to start Jupyter+Sage? how to discover an already running Jupyter? how to authenticate? Using a remote anonymous service as is done in this ticket is roughly the best we can hope to do.

Anyway, Vincent, I propose that we brainstorm about it over the phone. We are not converging here.

comment:28 Changed 3 years ago by nthiery

I have worked on improving the UI to give stronger feedback and background about the service to the user. You can try it on:

http://sage-package.readthedocs.io/en/latest/sage_package/sphinx-demo.html

comment:29 Changed 3 years ago by git

  • Commit changed from b4497c130de334ebe2f651e41c5301b754770406 to 4662206eef2926955c30c76a63f67b77a012cbc2

Branch pushed to git repo; I updated commit sha1. New commits:

4662206Merge branch 'develop' into t/24593/upgrade_from_thebe_to_thebelab_for_live_documentation_support

comment:30 Changed 2 years ago by git

  • Commit changed from 4662206eef2926955c30c76a63f67b77a012cbc2 to d50603a55316c83eb83a0235af63291ee795dfc3

Branch pushed to git repo; I updated commit sha1. New commits:

d50603a24593: fetch thebelab and run computation locally when possible

comment:31 Changed 2 years ago by saraedum

  • Status changed from needs_review to needs_work

There seem to be merge conflicts.

comment:32 Changed 2 years ago by git

  • Commit changed from d50603a55316c83eb83a0235af63291ee795dfc3 to 2781fe0c6e9b5bd3cb9a2e4421e2104561f43b39

Branch pushed to git repo; I updated commit sha1. New commits:

2781fe0Merge branch 'develop' into t/24593/upgrade_from_thebe_to_thebelab_for_live_documentation_support

comment:33 Changed 2 years ago by git

  • Commit changed from 2781fe0c6e9b5bd3cb9a2e4421e2104561f43b39 to 10437f012ab92a854b7163a9ad8afb8b8af1d826

Branch pushed to git repo; I updated commit sha1. New commits:

10437f024593: setup new thebelab spkg

comment:34 Changed 2 years ago by git

  • Commit changed from 10437f012ab92a854b7163a9ad8afb8b8af1d826 to dfe5d5c7cda4830fc836b7a23c2b85e9334b0de2

Branch pushed to git repo; I updated commit sha1. New commits:

dfe5d5c24593: minor fixes and improvements to the SPKG.txt

comment:35 Changed 2 years ago by git

  • Commit changed from dfe5d5c7cda4830fc836b7a23c2b85e9334b0de2 to d697404b6b8ccf35979431f48a3d1912190b1831

Branch pushed to git repo; I updated commit sha1. New commits:

d69740424593: there is only an order dependency with the Jupyter notebook

Changed 2 years ago by nthiery

comment:36 Changed 2 years ago by nthiery

With this new version:

  • thebelab is packaged for Sage, and added in the jupyter notebook extensions
  • if the page is served through a jupyter server, this tries to get thebelab from and run computations on that server; in particular, if the page is served from the users' SageMath jupyter (e.g. the Help button), no internet connection is required
  • if thebelab is not available locally, resort to downloading from internet
  • if there is no jupyter server or no sagemath kernel, resort to using binder

comment:37 Changed 2 years ago by nthiery

  • Status changed from needs_work to needs_review

comment:38 follow-up: Changed 2 years ago by nthiery

Vincent: I believe this is what we had agreed upon, right?

Questionable point

As a new package, should the thebelab spkg be standard or optional? Officially we should wait one year; however it replaces an existing package, and it should be rather failsafe (a simple js file to be installed by jupyter); also, up to the dependency upon on the network, the live doc would still work without it.

What do you think?

Timing

when browsing the doc through Jupyter, it takes 4s on my machine from pressing the Activate button to displaying the first output.

comment:39 Changed 2 years ago by nthiery

The result in action: http://sage-package.readthedocs.io/en/latest/sage_package/sphinx-demo.html

Of course, this does not showcase the use of a local jupyter server. That takes an installation ...

Changed 2 years ago by nthiery

comment:40 Changed 2 years ago by nthiery

For reference:

I made a quick experiment of a semi-automatic notebook emulating a static web page for testing whether thebelab works; it does not require the documentation to be compiled. See attached file (contains usage instructions). Maybe it could eventually be fully automated and integrated in our test suite.

comment:41 in reply to: ↑ 38 Changed 2 years ago by vdelecroix

Hi Nicolas,

Sorry to have not worked on this! Too busy at MathExp and now I need to take care about my moving from Bordeaux.

Replying to nthiery:

Vincent: I believe this is what we had agreed upon, right?

Looks like it! Though I am unsure about the 100% availability without internet. I will try.

Questionable point

As a new package, should the thebelab spkg be standard or optional? Officially we should wait one year; however it replaces an existing package, and it should be rather failsafe (a simple js file to be installed by jupyter); also, up to the dependency upon on the network, the live doc would still work without it.

For what and why should we wait one year? If it fixes issues it is fine to make it standard from now. Though this has to be discussed on sage-devel.

What do you think?

Timing

when browsing the doc through Jupyter, it takes 4s on my machine from pressing the Activate button to displaying the first output.

4s in which mode (local or mybinder)? Starting a Jupter worksheet is much faster, most of the time is spent in starting Sage (1.5secs).

comment:42 follow-up: Changed 2 years ago by vdelecroix

  • Status changed from needs_review to needs_work

The tarball in attachment is invalid...

[thebelab-0.2.1] Found local metadata for thebelab-0.2.1
[thebelab-0.2.1] Invalid checksum for cached file /opt/sage/upstream/thebelab-0.2.1.tar.bz2, deleting

comment:43 follow-up: Changed 2 years ago by vdelecroix

The instruction to build a new tarball

    THEBELAB_VERSION=0.2.1
    echo $THEBELAB_VERSION > package-version.txt
    mkdir src
    wget https://unpkg.com/thebelab@\^$THEBELAB_VERSION -O src/thebelab.js
    tar -jcvf thebelab-$THEBELAB_VERSION.tar.bz2 src
    rm  src/thebelab.js; rmdir src
    mv thebelab-$THEBELAB_VERSION.tar.bz2 ../../../upstream
    sage --package fix-checksum

should be in a spkg-src script (there are many of them in Sage for inspiration).

comment:44 follow-up: Changed 2 years ago by vdelecroix

Still in SPKG.txt I would prefer if #20690 and #24593 were complete url. It is not clear that you are refering to trac ticket numbers.

comment:45 in reply to: ↑ 42 Changed 2 years ago by vdelecroix

  • Description modified (diff)

Replying to vdelecroix:

The tarball in attachment is invalid...

[thebelab-0.2.1] Found local metadata for thebelab-0.2.1
[thebelab-0.2.1] Invalid checksum for cached file /opt/sage/upstream/thebelab-0.2.1.tar.bz2, deleting

Actually, the link to the tarball should be

https://trac.sagemath.org/raw-attachment/ticket/24593/thebelab-0.2.1.tar.bz2

(checksum were correct)

Last edited 2 years ago by vdelecroix (previous) (diff)

comment:46 follow-up: Changed 2 years ago by vdelecroix

Works well on my computer (and it is much faster than 4secs to start).

comment:47 follow-up: Changed 2 years ago by vdelecroix

Once you click on activate, two links appear in place of the button: "Sage" (pointing to sagemath.org) and "About" (pointing to http://sage-package.readthedocs.io/en/latest/sage_package/thebe.html). The latter one is pointing to a page that does not mention the fact that the examples can (and does) run with a local Jupyter.

comment:48 in reply to: ↑ 47 Changed 2 years ago by nthiery

Replying to vdelecroix:

Once you click on activate, two links appear in place of the button: "Sage" (pointing to sagemath.org) and "About" (pointing to http://sage-package.readthedocs.io/en/latest/sage_package/thebe.html). The latter one is pointing to a page that does not mention the fact that the examples can (and does) run with a local Jupyter.

Documentation pages produced with sage-package take further steps to reduce the dependence on network and cloud services. If the page is served by a Jupyter server, it will attempt to use it to fetch the Thebe javascript and run the computations.

That being said, you are welcome to edit the sources of this page if you have ideas to improve the wording and make the information more prominent while keeping it reasonably concise.

comment:49 in reply to: ↑ 43 Changed 2 years ago by nthiery

Replying to vdelecroix:

The instruction to build a new tarball

    THEBELAB_VERSION=0.2.1
    echo $THEBELAB_VERSION > package-version.txt
    mkdir src
    wget https://unpkg.com/thebelab@\^$THEBELAB_VERSION -O src/thebelab.js
    tar -jcvf thebelab-$THEBELAB_VERSION.tar.bz2 src
    rm  src/thebelab.js; rmdir src
    mv thebelab-$THEBELAB_VERSION.tar.bz2 ../../../upstream
    sage --package fix-checksum

should be in a spkg-src script (there are many of them in Sage for inspiration).

I considered this, but then dropped the idea after reading the dev guide: we are indeed not changing the upstream sources: ` The spkg-src file is optional and only to document how the upstream tarball was changed. Ideally it is not modified, then there would be no spkg-src file present either. `

Is your argument that building a tarball is a change by itself to the upstream sources? Then, yeah, sure, we could do that.

comment:50 Changed 2 years ago by git

  • Commit changed from d697404b6b8ccf35979431f48a3d1912190b1831 to b9b3ca0ba85056eda67f7b014d5d45e2bb7c167d

Branch pushed to git repo; I updated commit sha1. New commits:

b9b3ca024593: full links to the relevant tickets

comment:51 in reply to: ↑ 44 Changed 2 years ago by nthiery

Replying to vdelecroix:

Still in SPKG.txt I would prefer if #20690 and #24593 were complete url. It is not clear that you are refering to trac ticket numbers.

Done. That being said it seems that most SPKG.txt refer to ticket by number rather than URL ...

comment:52 in reply to: ↑ 46 Changed 2 years ago by nthiery

Works well on my computer (and it is much faster than 4secs to start).

Starting Sage by itself on my machine takes 2.5s :-) Indeed, if I am quick at clicking Activate -> Run, I can get down to 3s.

I guess it's going to be time to get a faster machine!

comment:53 Changed 2 years ago by slelievre

Note: there is popular demand for this feature!

See this question.posted today on StackOverflow:

comment:54 in reply to: ↑ description Changed 2 years ago by jdemeyer

Replying to nthiery:

thebelab tarball

https://trac.sagemath.org/raw-attachment/ticket/24593/thebelab-0.2.1.tar.bz2

Why is that an attachment on a Sage Trac ticket? Does thebelab not have proper releases?

comment:55 Changed 2 years ago by jdemeyer

  • Summary changed from Upgrade from Thebe to ThebeLab for live documentation support to Replace Thebe by ThebeLab for live documentation support

Does this require an installation of JupyterLab??

comment:56 follow-up: Changed 2 years ago by jdemeyer

This is not acceptable:

To upgrade the sources to a new version:

    THEBELAB_VERSION=0.2.1
    echo $THEBELAB_VERSION > package-version.txt
    mkdir src
    wget https://unpkg.com/thebelab@\^$THEBELAB_VERSION -O src/thebelab.js
    tar -jcvf thebelab-$THEBELAB_VERSION.tar.bz2 src
    rm  src/thebelab.js; rmdir src
    mv thebelab-$THEBELAB_VERSION.tar.bz2 ../../../upstream
    sage --package fix-checksum

If you really require a special script for generating the source tarball (why?), then it should be a spkg-src file. See build/pkgs/glpk/spkg-src for a good example.

comment:57 in reply to: ↑ 56 ; follow-up: Changed 2 years ago by vdelecroix

Replying to jdemeyer:

This is not acceptable:

To upgrade the sources to a new version:

    THEBELAB_VERSION=0.2.1
    echo $THEBELAB_VERSION > package-version.txt
    mkdir src
    wget https://unpkg.com/thebelab@\^$THEBELAB_VERSION -O src/thebelab.js
    tar -jcvf thebelab-$THEBELAB_VERSION.tar.bz2 src
    rm  src/thebelab.js; rmdir src
    mv thebelab-$THEBELAB_VERSION.tar.bz2 ../../../upstream
    sage --package fix-checksum

If you really require a special script for generating the source tarball (why?), then it should be a spkg-src file. See build/pkgs/glpk/spkg-src for a good example.

You are repeating comment:43...

comment:58 in reply to: ↑ 57 ; follow-up: Changed 2 years ago by nthiery

Replying to vdelecroix:

You are repeating comment:43...

Indeed. And in an unpleasant tone, especially given the ambiguity of the doc on the topic.

But let's leave this aside: I am totally fine with having a spkg-src script. Not sure I'll get to it right away though, so feel free to take that part over.

About thebelab's tarball: yes, AFAIK thebelab does not yet have an official tarball: it's released as a single javascript file distributed on usual CDN's; I guess this is a rather standard thing in the javascript world, but I am no expert in those things. I'll ask Min when I see him on Wednesday, and whether there should be an official tarball.

About the dependency on jupyterlab: yes thebelab uses the jupyterlab code base, but the relevant part is automatically bundled in the downloaded javascript. So it does not depend on jupyterlab being installed separately. Of course we may hope that, once Sage will ship by default with jupyterlab, there would be a way to not ship the same code twice.

Last edited 2 years ago by nthiery (previous) (diff)

comment:59 in reply to: ↑ 58 Changed 2 years ago by jdemeyer

Replying to nthiery:

And in an unpleasant tone

Sorry for that, I didn't intend that.

About thebelab's tarball: yes, AFAIK thebelab does not yet have an official tarball: it's released as a single javascript file distributed on usual CDN's; I guess this is a rather standard thing in the javascript world, but I am no expert in those things. I'll ask Min when I see him on Wednesday, and whether there should be an official tarball.

Please ask him. There are also the github tarballs (https://github.com/minrk/thebelab/releases), would that work?

About the dependency on jupyterlab: yes thebelab uses the jupyterlab code base, but the relevant part is automatically bundled in the downloaded javascript. So it does not depend on jupyterlab being installed separately.

Thanks for the answer. It would be good to document this clearly in SPKG.txt.

comment:60 Changed 2 years ago by nthiery

Brief update after talking to min: thebelab is being distributed through npm, and a tarball can be obtained from:

https://registry.npmjs.org/thebelab/-/thebelab-0.2.1.tgz

I'll try to update the branch here accordingly, unless someone is quicker!

Last edited 2 years ago by nthiery (previous) (diff)

comment:61 Changed 2 years ago by git

  • Commit changed from b9b3ca0ba85056eda67f7b014d5d45e2bb7c167d to e635c1c2392b8be603c155d89adcebb712106c9e

Branch pushed to git repo; I updated commit sha1. New commits:

75b2b3e24593: fetched official tarball, and updated update instructions
e635c1c24593: improved wording

comment:62 Changed 2 years ago by nthiery

  • Description modified (diff)
  • Status changed from needs_work to needs_review

comment:63 Changed 2 years ago by git

  • Commit changed from e635c1c2392b8be603c155d89adcebb712106c9e to f0b2a243ba59e449090a1d19e0ad18fb2ced2949

Branch pushed to git repo; I updated commit sha1. New commits:

f0b2a2424593: explanations on the (non) dependency on jupyterlab

comment:64 Changed 2 years ago by nthiery

Alright; I guess all the reviewers suggestions have been addressed?

Cheers

comment:65 follow-up: Changed 2 years ago by nthiery

  • Status changed from needs_review to needs_work
  • Work issues set to tarball format changed

Oops, the format of the tarball downloaded from npm is not the same as the one I was creating myself ...

comment:66 Changed 2 years ago by git

  • Commit changed from f0b2a243ba59e449090a1d19e0ad18fb2ced2949 to fa965b97348d5c9231d7d4853f0d704f96923f03

Branch pushed to git repo; I updated commit sha1. New commits:

fa965b9Merge branch 'develop' into t/24593/upgrade_from_thebe_to_thebelab_for_live_documentation_support

comment:67 Changed 2 years ago by git

  • Commit changed from fa965b97348d5c9231d7d4853f0d704f96923f03 to a0f40172c6d66b415c3ea9d1a22a958601ec8e33

Branch pushed to git repo; I updated commit sha1. New commits:

a0f401724593: spkg-src script to download the webpacked'd version of thebelab

comment:68 Changed 2 years ago by git

  • Commit changed from a0f40172c6d66b415c3ea9d1a22a958601ec8e33 to 99ad250327d69e146c7df63554f9b7b64fe35d79

Branch pushed to git repo; I updated commit sha1. New commits:

99ad25024593: make spkg-install executable

comment:69 Changed 2 years ago by nthiery

  • Status changed from needs_work to needs_info

The bundled spkg-src script builds the tarball from npm; this way we delegate to npm the building (with webpack) of the standalone js file for thebelab.

comment:70 Changed 2 years ago by nthiery

  • Status changed from needs_info to needs_review

comment:71 Changed 2 years ago by nthiery

  • Work issues tarball format changed deleted

comment:72 Changed 2 years ago by git

  • Commit changed from 99ad250327d69e146c7df63554f9b7b64fe35d79 to 02fce9d82063781e7bad8d825c73bca9c1574d2c

Branch pushed to git repo; I updated commit sha1. New commits:

02fce9dMerge branch 'develop' into t/24593/upgrade_from_thebe_to_thebelab_for_live_documentation_support

comment:73 Changed 2 years ago by git

  • Commit changed from 02fce9d82063781e7bad8d825c73bca9c1574d2c to 551fd430146db6701d553ed5b8031e6440b3302f

Branch pushed to git repo; I updated commit sha1. New commits:

551fd4324593: adds missing spkg-src

Changed 2 years ago by nthiery

comment:74 Changed 2 years ago by nthiery

  • Description modified (diff)

comment:75 in reply to: ↑ 65 ; follow-up: Changed 2 years ago by jdemeyer

  • Status changed from needs_review to needs_info

Replying to nthiery:

Oops, the format of the tarball downloaded from npm is not the same as the one I was creating myself ...

Sorry, what you mean with this? In other words: why cannot you use the tarball ​https://registry.npmjs.org/thebelab/-/thebelab-0.2.1.tgz

There also seems to be a version mismatch: on the ticket, you posted thebelab-0.2.1.2.tgz (I assume created using spkg-src) but in the branch, there is version 0.2.1

comment:76 Changed 2 years ago by jdemeyer

Typo: shiped -> shipped

comment:77 Changed 2 years ago by jdemeyer

Doesn't the ^ mean that the version might not be the exact version that you specified?

wget https://unpkg.com/$PACKAGENAME@\^$UPLOADVERSION -O src/$PACKAGENAME.js

By the way, this might be more readable as follows:

wget "https://unpkg.com/${PACKAGENAME}@${UPLOADVERSION}" -O src/$PACKAGENAME.js

comment:78 in reply to: ↑ 75 Changed 2 years ago by nthiery

Replying to jdemeyer:

Replying to nthiery:

Oops, the format of the tarball downloaded from npm is not the same as the one I was creating myself ...

Sorry, what you mean with this? In other words: why cannot you use the tarball ​https://registry.npmjs.org/thebelab/-/thebelab-0.2.1.tgz

Because it's the source tarball for thebelab. Oh, but actually there is a lib subdirectory which contained the webpacked's javascript file; I had missed that. So we could actually use it.

There also seems to be a version mismatch: on the ticket, you posted thebelab-0.2.1.2.tgz (I assume created using spkg-src) but in the branch, there is version 0.2.1

That was just me running through hoops to replace an existing file ... Anyway we don't need it any more.

comment:79 Changed 2 years ago by nthiery

I am on vacations until August 14th; feel free to proceed and implement the suggested changes. Otherwise I'll take care of them when I am back.

Thanks for the review

comment:80 Changed 2 years ago by jdemeyer

  • Description modified (diff)

comment:81 Changed 2 years ago by jdemeyer

  • Branch changed from u/nthiery/upgrade_from_thebe_to_thebelab_for_live_documentation_support to u/jdemeyer/upgrade_from_thebe_to_thebelab_for_live_documentation_support

comment:82 Changed 2 years ago by jdemeyer

  • Authors changed from Nicolas M. Thiéry to Nicolas M. Thiéry, Jeroen Demeyer
  • Commit changed from 551fd430146db6701d553ed5b8031e6440b3302f to ab1f2b43b52ad607720a492948e6fdec1b72f21a
  • Milestone changed from sage-8.2 to sage-8.5

New commits:

51762c424593: Upgrade from Thebe to ThebeLab for live documentation support
ab1f2b4Use thebelab tarball from npm

comment:83 Changed 2 years ago by jdemeyer

  • Description modified (diff)

comment:84 Changed 2 years ago by jdemeyer

  • Description modified (diff)

comment:85 Changed 2 years ago by git

  • Commit changed from ab1f2b43b52ad607720a492948e6fdec1b72f21a to c9910413af226d145632e8a1bb7b7e0dd4a67720

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

c991041Use thebelab tarball from npm

comment:86 Changed 2 years ago by jdemeyer

This is working as intended, both locally and remotely, except that it's always downloading the font from the web.

comment:87 Changed 2 years ago by jdemeyer

I found out that the font comes from jupyterlab.

comment:88 follow-up: Changed 2 years ago by nthiery

Ok. So presumably we will have that font available locally as soon as Jupyterlab will ship with Sage. I would tend to see this as a very minor caveat, given that the whole thing works without Internet, albeit with a slightly less nice font. So, unless we see an immediate solution, I would tend to proceed as is for now.

Other than this font issue, is there anything left to be done? Or shall I press positive review once I'll have reviewed your latest changes?

comment:89 Changed 2 years ago by jdemeyer

  • Status changed from needs_info to needs_review

comment:90 in reply to: ↑ 88 Changed 2 years ago by jdemeyer

Replying to nthiery:

Other than this font issue, is there anything left to be done?

I don't think so.

comment:91 Changed 2 years ago by slelievre

For reference:

  • #26059 Make JupyterLab an optional package
  • #24904 Make JupyterLab a standard package

comment:92 Changed 2 years ago by jdemeyer

ping?

comment:93 Changed 7 months ago by chapoton

  • Keywords notebook added
  • Milestone changed from sage-8.5 to sage-9.2
  • Status changed from needs_review to needs_work

red branch and OpenDreamKit? is over

comment:94 Changed 4 months ago by mkoeppe

Could someone please take care of this ticket before it bitrots completely?

comment:95 Changed 3 months ago by mkoeppe

  • Cc jhpalmieri klee added

comment:96 Changed 3 months ago by slelievre

In the meantime Thebelab changed its name to Thebe.

For reference the renaming discussion is here:

Last edited 3 months ago by slelievre (previous) (diff)

comment:97 Changed 3 months ago by mkoeppe

... but we still have the old Thebe?

comment:98 Changed 3 months ago by mkoeppe

This seems to be upstream now: https://github.com/executablebooks/thebe

comment:100 Changed 6 weeks ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3
Note: See TracTickets for help on using tickets.