Add support for uploading static html doc page as a worksheet in the notebook

[nthiery]

This patch adds (experimental) support for uploading a static html doc page as a worksheet in the notebook. As a side effect, it:

• allows for uploading a .txt file from a URL (was broken)
• allows for uploading a zip file containing .txt and .html files

This little feature would give a very simple workflow for distributing tutorial worksheets during our Sage(-Combinat) workshops, saving a lot of hassle and time to both us and the participants (more detail upon request). It would be great if this feature could be in the sage stable release by early May for the next Sage-Combinat workshop in Acadia.

I wrote the attached small patch as a proof of concept; however this is the first time I am touching the notebook code and I am not planning to invest in this direction in the short term. So I would be extremely glad if some notebook expert could take over this patch, clean it up (look for TODO/FIXME), and rebase it upon #10521 with whom it probably conflicts lightly (this should be a breeze).

Little issues:

• url's ending with something like #blah are not supported, whereas our current setup creates links with such. Maybe the #blah part could be stripped out. Or we should fix our setup :-)
• The produced worksheet does not look as nice as a usual worksheet or the live documentation (in particular, there are some useless navigation links on top). That's probably a question of style file. Or maybe the navigation links should be stripped out.

Future extensions:

• Add support for attachements, non text outputs, etc (they are currently lost during the upload)

Related tickets:

• #4825: Uploading a pdf which contains an attached .sws will extract the .sws
• #7441: notebook: make it possible to upload from the url of a published worksheet
• #9875: Can't upload from a notebook link, only from a .sws file
• #10521: notebook upload of zipped worksheets can't deal with multiple worksheets with the same name

Thanks so much in advance!!!

Apply: trac_10652-upload-static-documentation-as-worksheet-nt.patch in the sagenb repository

Nicolas

[jason]

Added #4825 to the list of related tickets. That ticket proposes that uploading pdf files with attached .sws files would extract the .sws file and put it in the users list of worksheets. I could see this also being useful for tutorials, books, articles, etc.

[jason]

This little feature would give a very simple workflow for distributing tutorial worksheets during our Sage(-Combinat) workshops, saving a lot of hassle and time to both us and the participants (more detail upon request).

I'm curious about how you plan on using this. I'd love an easy way to distribute lots of tutorials.

[hivert]

+1 !!! We badly need it !!!

Florent

[nthiery]

Hi Jason!

I'm curious about how you plan on using this. I'd love an easy way to distribute lots of tutorials.

So much for being lazy :-) Here it is!

We write our tutorials as rst files in the reference manual (exact location under discussion), and share them using our Sage-Combinat patch server ([1]). On any server, we compile the html documentation, and make it available to the participants ([2]).

Finaly, we point the participants to [3]. They can then click on any of the static tutorials; if they like it, they can just copy paste the url [4] from their brower into the "Upload" field of their notebook.

Note: in case the server runs a notebook server, or if one has the sage-combinat patches installed, then one can directly use the live documentation.

Cheers,

Nicolas

[1] ​http://combinat.sagemath.org/hgwebdir.cgi/patches/file/tip/sage-demos-and-tutorials-nt.patch [2] ​http://combinat.sagemath.org/doc/reference/demos/ [3] ​http://combinat.sagemath.org/doc/reference/demos/2011-01-17-SageDays28.html [4] ​http://combinat.sagemath.org/doc/reference/demos/demo-algebraic-combinatorics-short.html

[ddrake]

This is great work. It should definitely be possible to upload anything sensible to the notebook and have it give you a worksheet.

Given the upcoming changes in the notebook, perhaps this ticket should be viewed as a feature request for the new Flask-based notebook.

[nthiery]

Hi Dan!

Replying to ddrake:

This is great work.

:-)

Given the upcoming changes in the notebook, perhaps this ticket should be viewed as a feature request for the new Flask-based notebook.

This sounds natural indeed!

However, since the feature is needed very soon, and the change minor, I still would appreciate if this patch (or some improved version) was merged in soon in the current notebook.

[kcrisman]

Hi nthiery,

This would be a nice change. See also ​sws2tex and ​tex2sws for related projects.

I am sorry that I can't review this and test it properly right now, though the code seems ok in the main. What does the "assert False" do - I know what assert usually does, but I assume this is some special use of that statement?

Also, it would perhaps be better to us os.path or something like that

sage: a = "http://www.me.com/mysheet.sws"
sage: os.path.splitext(a)
('http://www.me.com/mysheet', '.sws')

in twist.py and then just check the result[1] for the extensions that are allowed all at once, keeping that for glueing on later, also with os.path?

By the way, you could perhaps just use this on your combinat server - sometimes people put custom patches on servers, it's not hard to do if you're the admin. That would keep it from being quite as critical to the rest of Sage.

[nthiery]

Replying to kcrisman:

This would be a nice change. See also ​sws2tex and ​tex2sws for related projects.

I am sorry that I can't review this and test it properly right now, though the code seems ok in the main. What does the "assert False" do - I know what assert usually does, but I assume this is some special use of that statement?

Just a big fat advertisement that the execution flow is not supposed to get there :-)

A proper assertion should be raised instead, but as the comment states, I would need to have some feed back from the notebook people to know which assertion would be most appropriate here.

Ok, I guess I'll just raise an RunTimeError? for the moment.

Also, it would perhaps be better to us os.path or something like that

sage: a = "http://www.me.com/mysheet.sws"
sage: os.path.splitext(a)
('http://www.me.com/mysheet', '.sws')

in twist.py and then just check the result[1] for the extensions that are allowed all at once, keeping that for glueing on later, also with os.path?

Will do now.

By the way, you could perhaps just use this on your combinat server - sometimes people put custom patches on servers, it's not hard to do if you're the admin. That would keep it from being quite as critical to the rest of Sage.

The point is that we want to use this feature with the very first tutorials. In particuliar with complete newbies. If using Sage starts by having to apply a patch, we are going to scare them away :-) And many of them won't have the required development tools to install the Sage-Combinat queue, etc.

Thanks for your comments,

Cheers,

Nicolas

[kcrisman]

I see, you mean you want them to be able to upload them to their own local Sage installations or servers. I understand.

I'd raise some sort of input error or value error myself, with message about the URL file ending not one that we support (perhaps with a list? though such lists can get long as we support more and more extensions...).

Again, sorry I can't do a full review. Can any of the other sage-combinat people do it? Of course, even if we made a new sagenb spkg (perhaps with jmol and a couple other things that are about ready) there is no guarantee that Jeroen will put it in 4.7, since he already has made four alphas.

[jason]

The code mainly looks okay to me, but here are a few points that worry me. I haven't tested this, so this is not a full review (yet).

1. We of course don't want the notebook to crash if someone uploads a file other than the right type. The assert statement in checking this worries me. I'd say a ValueError? would be appropriate (we got something that was the wrong value).

2. In the _import_worksheet_html function, there is a return statement, but then there are two more lines of code. What are those two lines doing after the return statement?

[kcrisman]

Replying to jason:

The code mainly looks okay to me, but here are a few points that worry me. I haven't tested this, so this is not a full review (yet).

1. We of course don't want the notebook to crash if someone uploads a file other than the right type. The assert statement in checking this worries me. I'd say a ValueError? would be appropriate (we got something that was the wrong value).

So that's what assert False would do? Okay, then I definitely would raise an error. Jason's suggestion sounds fine.