#11459 closed enhancement (fixed)
Implement sage -rst2txt and sage -rst2sws
Reported by: | slabbe | Owned by: | slabbe |
---|---|---|---|
Priority: | major | Milestone: | sage-5.1 |
Component: | notebook | Keywords: | sd31 days31 sd40.5 |
Cc: | kcrisman, kini, jasonbhill | Merged in: | sage-5.1.beta2 |
Authors: | Sébastien Labbé | Reviewers: | Keshav Kini |
Report Upstream: | N/A | Work issues: | |
Branch: | Commit: | ||
Dependencies: | #11080 | Stopgaps: |
Description (last modified by )
This ticket concerns the following :
- Convert a ReST file into a worksheet file from the command line :
sage -rst2txt
. - Convert a ReST file into a Sage worksheet from the command line :
sage -rst2sws
. - Upload a ReST (.rst) file from the Notebook.
- Upload a .html file output from docutils rst2html on a ReST.
- Upload a
worksheet.html
file as those stored in the repository~/.sage/sage_notebook.sagenb/home/admin
without having to change the extension to.txt
. - Add the class
docutilsHTMLProcessor
to the filesagenb/notebook/docHTMLProcessor.py
which translates HTML file generated by docutilsrst2html
command into a worksheet text file. This class was inspired from the class
SphinxHTMLProcessor
already in the same Python file. - Add the class
genericHTMLProcessor
to the same file which gathers all the common methods of the other two classes. - Improve a lot the documentation of the file
sagenb/notebook/docHTMLProcessor.py
: it is now 100% doctest coveraged.
My motivation is that in general I wrote my own code using the Sage Code Convention where examples are formatted in ReST. The possibility of converting rst syntax to a worksheet would be nice. Also, ReST file is more fun to edit than the html/cells worksheet text file. It is also more versatile as it can be used to generate a web page or S5 presentation, etc. using docutils.
On my web page, I created a page rst2sws where I put an example of ReST file. Note that I managed to use MathJax to render the HTML output of docutils rst2html
command.
Here is my clone of the sage-flask-notebook where I my patch on the sage notebook will be available.
I am planning to make this script more clean during Sage Days 31. I will upload patches as it gets better.
Related to : #10637 (Implement sage -sws2rst) and #10652.
I mentionned the idea on sage-devel in December 2010 here.
This is based on a script called rst2notebook
I wrote in January 2011 available on the Sage Combinat server.
---
Apply: trac_11459-root.patch, trac_11459-root.reviewer.patch to $SAGE_ROOT
Apply: trac_11459-scripts.patch, trac_11459-scripts.reviewer.patch to $SAGE_LOCAL/bin
Attachments (6)
Change History (48)
comment:1 Changed 11 years ago by
- Keywords sd31 days31 added
comment:2 follow-up: ↓ 3 Changed 11 years ago by
- Cc kcrisman added
comment:3 in reply to: ↑ 2 ; follow-up: ↓ 4 Changed 11 years ago by
Replying to kcrisman:
Maybe this should be rst2sws rather than rst2notebook? By analogy with #10637 sws2rst.
By rst2notebook, I mean something different. I mean the worksheet txt file which is made of html code and {{{
's cells. I agree rst2notebook is confusing. Maybe sage -rst2worksheet_txt
would be better?
By the way, I also implemented the sage -rst2sws
. I still need to add some doctests and do some more testing before it needs review. I hope to get to that point by wednesday.
Sébastien
comment:4 in reply to: ↑ 3 Changed 11 years ago by
Replying to slabbe:
Replying to kcrisman:
Maybe this should be rst2sws rather than rst2notebook? By analogy with #10637 sws2rst.
By rst2notebook, I mean something different. I mean the worksheet txt file which is made of html code and
{{{
's cells. I agree rst2notebook is confusing. Maybesage -rst2worksheet_txt
would be better?
Hmm, or something like that. That is really long. But rst2worksheet would be misleading as well, so maybe that's the best option.
What about rst2txt?
By the way, I also implemented the
sage -rst2sws
. I still need to add some doctests and do some more testing before it needs review. I hope to get to that point by wednesday.
Great! Except for the fact that I usually need to go the other direction - but of course those are also appearing.
comment:5 Changed 11 years ago by
What about rst2txt?
Well, at first, I excluded that because txt extension are often used for writing ReST files. But, on the other hand, when one uploads a file having a .txt
extension from the notebook, it is assume to be that worksheet txt file made of html code and triple braces cells. So I think you are right. rst2txt is a good suggestion. Thank you!
By the way, I also implemented the
sage -rst2sws
. I still need to add some doctests and do some more testing before it needs review. I hope to get to that point by wednesday.Great! Except for the fact that I usually need to go the other direction - but of course those are also appearing.
Why BTW? I feel I am the only one that needs more the rst -> sws way?! I write all my examples in ReST syntax in my personal code, that might be why.
comment:6 Changed 11 years ago by
- Description modified (diff)
- Summary changed from Implement sage -rst2notebook to Implement sage -rst2txt and sage -rst2sws
comment:7 Changed 11 years ago by
- Cc kini added
comment:8 Changed 11 years ago by
- Cc jasonbhill added
comment:9 Changed 11 years ago by
- Description modified (diff)
comment:10 Changed 11 years ago by
- Description modified (diff)
comment:11 Changed 11 years ago by
I'm testing this, but I wanted to give others a heads-up on something. (Maybe Kini can shed some light on this from the hg side of things.)
I'm currently using the flask repo (the ~official~ one...) and have a symlink redirecting devel/sagenb->flask. The path for various diffs in this patch should be following that sagenb symlink, but they're following my sage->sage-main (or whatever clone I'm working in) repo instead.
comment:12 Changed 11 years ago by
The root of the sagenb repository is in your devel/sagenb
, i.e. already beyond the symlink. The paths should have nothing to do with that (and look fine to me fwiw). Are you applying it in the right place?
comment:13 Changed 11 years ago by
Yup... "sagenb" and "sage" are the two symlinks in devel. All of my patches are attempting to following the latter.
I'm using hg_sage.apply("address"). My usual workflow is to create a clone, build into that clone, then apply any patches. This has always worked, but then again I am always working under the "sage" symlink. But, in this situation the patch (regardless of branch) is attempting to apply in "devel/sage->my_".
How might one apply this otherwise ... in a specific location?
comment:14 Changed 11 years ago by
Ah. Use hg_sagenb
instead of hg_sage
- try tab completing hg_
in the Sage console to see the list of repositories you have access to. hg_sage
is the Sage library, hg_sagenb
is the notebook, hg_extcode
seems to be for the interfaces to other software, hg_scripts
is for the sage command line tools and related stuff, hg_root
is for some miscellaneous "top-level" stuff.
I would personally recommend using Mercurial from the command line, though - it allows you to see what you are doing more precisely, where patches are going exactly, etc.
comment:15 Changed 11 years ago by
To be precise, hg_sage
points to devel/sage
, hg_sagenb
points to devel/sagenb
, hg_extcode
points to data/extcode
, and hg_scripts
points to local/bin
, if you want to go exploring. Using sage -clone
and sage -b <branchname>
only apply to devel/sage
- other repositories are left uncloned and unmodified.
By the way, you might consider dropping by IRC to talk about these things in realtime instead of posting to this trac ticket, as this is getting quite off-topic... :)
comment:16 follow-up: ↓ 18 Changed 11 years ago by
Salut!
Just to give an update about the status of the patches on this ticket :
- trac11459_rst2notebook_sagenb-sl.patch : I was not able to export the patch once I qfinish it and pushed it on my branch on the sagenb google code. Hence, the patch available here do not have the very last modification I did just before I made qfinish. So if you are testing it, it would be better if you test my sagenb branch. Anyhow, I do not intend to change my sagenb branch. I believe it is ready for review.
Question for kini : How can I export the patch once it is qfinished (commited) so that I upload here the latest version?
- trac11459_rst2notebook_localbin-sl.patch : The script needs work. I want to implement better options about escaping backslashes. I should be able to finish this in the next days during the week. Once this is done, I will change the status of this ticket to needs review.
Sébastien
comment:17 Changed 11 years ago by
You can turn committed changes back into patches with hg qimport -r tip
if you want. If you want to leave them committed but export a patch, use hg export tip > file.patch
. (don't forget to put "[diff]\ngit=yes" in your ~/.hgrc
.)
comment:18 in reply to: ↑ 16 Changed 11 years ago by
Hence, the patch available here do not have the very last modification I did just before I made qfinish.
It was not true finally. The patch trac11459_rst2notebook_sagenb-sl.patch available here contains the very last modifications.
comment:19 Changed 11 years ago by
- Status changed from new to needs_review
I finished working on the local/bin
scripts and just re-uploaded the patch.
Now needs review!
comment:20 Changed 10 years ago by
- Dependencies set to #11080
comment:21 Changed 10 years ago by
Hi Sebastien,
I've rebased the scripts patch to Sage 5.0. Because the file formerly at $SAGE_LOCAL/bin/sage-sage
is now at $SAGE_ROOT/spkg/bin/sage
, it had to be split into two patches.
The notebook patch will probably need to be rebased as well, but since we're doing development of sagenb on github now, it won't go here on trac. I'll see what I can do...
comment:22 Changed 10 years ago by
- Milestone changed from sage-5.1 to sage-5.0
Never mind, it looks like you already got that code into sagenb after all.
comment:24 Changed 10 years ago by
- Keywords sd40.5 added
- Reviewers set to Keshav Kini
- Status changed from needs_review to positive_review
So here is a reviewer patch, and positive review for this ticket. Everything looks good to me.
comment:25 Changed 10 years ago by
- Keywords changed from sd31, days31, sd40.5 to sd31 days31 sd40.5
comment:26 Changed 10 years ago by
- Description modified (diff)
Whoops, forgot the other reviewer patch...
comment:27 Changed 10 years ago by
Hi Kini,
Thanks for looking at this!
I remember I wanted to remove the guillemets (") in the line of documentation of both scripts:
".. escape-backslashes"
Can you update your patch to do it, if you agree? I am sorry I never did it...
Is the new notebook now part of Sage ? If I want to test this ticket, should I install the latest sage-alpha or should I install the new sage-notebook?
Sébastien
comment:28 Changed 10 years ago by
- Status changed from positive_review to needs_work
OK, I'll do it, and I'll also modify the file sage-rst2txt
which I completely overlooked...!
The new notebook is not part of Sage yet (though it finally got positive review a couple of hours ago - #11080). If you want to test the ticket, follow the instructions on #11080 to install the new notebook first.
comment:29 Changed 10 years ago by
- Status changed from needs_work to positive_review
OK, there we go.
comment:30 Changed 10 years ago by
- Milestone changed from sage-5.1 to sage-pending
comment:31 Changed 10 years ago by
Thanks for the review Keshav!
comment:32 Changed 10 years ago by
- Merged in set to sage-5.1.beta2
- Resolution set to fixed
- Status changed from positive_review to closed
comment:33 Changed 10 years ago by
Hi! I'm trying to get #10637 (the other direction) in shape for positive review, and I notice there are no patches here documenting in the Sage library how to use this. I guess that's because they just make text and sws. Was there any place that you would think would be natural, other than sage -advanced
?
comment:34 Changed 10 years ago by
Was there any place that you would think would be natural, other than sage -advanced?
Note: There is some documentation in sage -rst2sws -h
as well.
It would be a good idea to add documentation somewhere in the Sage reference manual. Where are the other sage-scripts documented?
comment:35 Changed 10 years ago by
My sense is that they are not.
Okay, I'll look at your script and see how you implement that, then do that at #10637. That would suffice for the purposes there.
Do you want to open another ticket to document (at least some) of the scripts in the usual reference manual?
comment:36 Changed 10 years ago by
By the way, not that you have to deal with this now, see http://docs.python.org/library/optparse.html:
Deprecated since version 2.7: The optparse module is deprecated and will not be developed further; development will continue with the argparse module.
Oh well.
comment:37 Changed 10 years ago by
- Milestone changed from sage-pending to sage-5.1
comment:38 follow-up: ↓ 39 Changed 10 years ago by
This "depends" on #11080 but somehow got merged anyway.
comment:39 in reply to: ↑ 38 ; follow-up: ↓ 41 Changed 10 years ago by
Replying to jdemeyer:
This "depends" on #11080 but somehow got merged anyway.
It depends on code written in the new sagenb indeed. That means the script won't work on sage-5.1 : it will get an import error I would guess. The thing is that there is no test for the script... so you could not see that the script was broken without #11080. Or worse, in the future, if a patch on sagenb breaks the script, we won't know it. How should a script be doctested?
comment:40 Changed 10 years ago by
The part of the script which depends on the new sagenb is the following :
# Do the translation html -> worksheet txt from sagenb.notebook.docHTMLProcessor import docutilsHTMLProcessor translator = docutilsHTMLProcessor() worksheet_txt = translator.process_doc_html(html) # create a Notebook from sagenb.notebook.notebook import Notebook from sage.misc.misc import tmp_dir nb = Notebook(tmp_dir()+'.sagenb') # create a worksheet W = nb.create_new_worksheet(title, 'admin') W.edit_save(worksheet_txt) # export the worksheet as sws nb.export_worksheet(W.filename(), output, title=title) # delete the Notebook nb.delete()
Is it possible that such code get broken in future version of sagenb?
comment:41 in reply to: ↑ 39 ; follow-up: ↓ 42 Changed 10 years ago by
Replying to slabbe:
How should a script be doctested?
You can have a look at sage/tests/cmdline.py
for many examples.
In any case, such a test should be the subject of a new ticket.
comment:42 in reply to: ↑ 41 Changed 10 years ago by
In any case, such a test should be the subject of a new ticket.
This is now #13297.
Maybe this should be rst2sws rather than rst2notebook? By analogy with #10637 sws2rst.