Opened 23 months ago

Last modified 13 months ago

#28838 closed defect

Resurrect "sage -sws2rst" via a pip-installable package sage_sws2rst — at Version 48

Reported by: tmonteil Owned by:
Priority: critical Milestone: sage-9.2
Component: python3 Keywords:
Cc: vbraun, dimpase, kcrisman, jhpalmieri Merged in:
Authors: Frédéric Chapoton, Matthias Koeppe Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/28838 (Commits, GitHub, GitLab) Commit: 53997700ee5e86b06d62607dca70204cc12a6a44
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

An important feature is to still be able to handle old .sws worksheets because a lot of pedagogical material was written on that format.

With the removal of sagenb, also sws2rst disappeared, see #30076 and related tickets.

We extract and translate the required material from sagenb to a new optional package sage_sws2rst that makes sage -sws2rst work.

As of this ticket, the source code of sage_sws2rst is in build/pkgs/sage_sws2rst/src, which is a pip-installable Python package; and one can build an sdist from there, for example using the spkg-src script. The source tree could as well be split out to a separate repository.

See also:

Change History (48)

comment:1 Changed 23 months ago by fbissey

Is converting .sws file covered by sagenb_export?

comment:2 Changed 23 months ago by tmonteil

  • Cc vbraun added

It seems not directly: after a quick look, the script seems to parse the html page, but maybe it could try to parse the html that belongs to the .sws archive (not sure if it has the same structure). Let me CC Volker for a better understanding of what is doable.

comment:3 Changed 23 months ago by chapoton

  • Branch set to u/chapoton/28838
  • Commit set to 8b5de13003724601c70e36c1a11aa83db5fdaff8

Here is a first sketch. The moved files are not at the level of sage requirements, as many doctests are missing. If this matters for you, please take the job.


New commits:

8b5de13first sketch of sws2rst tools (moved from sagenb)

comment:4 Changed 23 months ago by tmonteil

Shouldn't this go to a subdirectory of src/ext/ (e.g. src/ext/sws2rst/) ?

comment:5 Changed 23 months ago by chapoton

Salut Thierry.

Feel free to put it anywhere you like. I am not going to do anything more here.

comment:6 Changed 22 months ago by embray

  • Milestone changed from sage-9.0 to sage-9.1

Ticket retargeted after milestone closed

comment:7 Changed 18 months ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

comment:8 Changed 14 months ago by mkoeppe

  • Cc dimpase added
  • Milestone changed from sage-9.2 to sage-duplicate/invalid/wontfix
  • Status changed from new to needs_review

outdated, sws stuff was removed in #30076

comment:9 follow-up: Changed 14 months ago by tmonteil

  • Milestone changed from sage-duplicate/invalid/wontfix to sage-9.3
  • Status changed from needs_review to needs_work

It is even more important now that there is no sagenb anymore, while there are still sws files that people might want to convert (e.g. from a former course). I will try to work on this.

comment:10 follow-up: Changed 14 months ago by dimpase

  • Milestone changed from sage-9.3 to sage-duplicate/invalid/wontfix
  • Reviewers set to Dima Pasechnik
  • Status changed from needs_work to positive_review

it needs sagenb, a non-starter.

comment:11 in reply to: ↑ 10 Changed 14 months ago by tmonteil

  • Milestone changed from sage-duplicate/invalid/wontfix to sage-9.2
  • Status changed from positive_review to needs_work

Replying to dimpase:

it needs sagenb, a non-starter.

It only needs a small part of it (see Frédéric's patch), and the feature is still needed.

comment:12 Changed 14 months ago by mkoeppe

Given that this has extra dependencies (beautifulsoup) that we have just removed, this should really be packaged as a pip-installable little package and NOT be put into sage.misc.

comment:13 Changed 14 months ago by mkoeppe

  • Summary changed from sage -sws2rst broken to Resurrect "sage -sws2rst" via a pip-installable package sage_sws2rst

comment:14 Changed 14 months ago by dimpase

I don't see why such an extraction cannot be carried out in a version of Sage where this tool still works.

comment:15 Changed 14 months ago by mkoeppe

It seems like a great idea to me to provide this converter - as long as this code is not dumped into sagelib.

comment:16 Changed 14 months ago by mkoeppe

I'll be happy to help with packaging

comment:17 in reply to: ↑ 9 Changed 14 months ago by mkoeppe

Replying to tmonteil:

It is even more important now that there is no sagenb anymore, while there are still sws files that people might want to convert (e.g. from a former course). I will try to work on this.

Any progress?

comment:18 Changed 14 months ago by mkoeppe

  • Branch changed from u/chapoton/28838 to u/mkoeppe/28838

comment:19 follow-up: Changed 14 months ago by mkoeppe

  • Commit changed from 8b5de13003724601c70e36c1a11aa83db5fdaff8 to a205ec244b29b75a83581c1df9876f1a3bf824ba

How is this different from sagenb_export by the way?


New commits:

c93de0bfirst sketch of sws2rst tools (moved from sagenb)
fa9fb28Move sws2rst code to pip-installable package build/pkgs/sage_sws2rst/src
a205ec2Fixup

comment:20 Changed 14 months ago by git

  • Commit changed from a205ec244b29b75a83581c1df9876f1a3bf824ba to 9464457106de708fe31d51806654b47fc19a677f

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

dc41e19first sketch of sws2rst tools (moved from sagenb)
fe60540Move sws2rst code to pip-installable package build/pkgs/sage_sws2rst/src
9464457Fixup

comment:21 Changed 14 months ago by git

  • Commit changed from 9464457106de708fe31d51806654b47fc19a677f to 0408e6063f7c2f919fb88edacb9f9a97c550afbc

Branch pushed to git repo; I updated commit sha1. New commits:

0408e60build/pkgs/sage_sws2rst: Add dependency, fix import

comment:22 Changed 14 months ago by git

  • Commit changed from 0408e6063f7c2f919fb88edacb9f9a97c550afbc to 3acd1dd4466dcde1303982b00e55ba53bd2af2e4

Branch pushed to git repo; I updated commit sha1. New commits:

3acd1ddsrc/bin/sage: Restore --sws2rst option

comment:23 Changed 14 months ago by mkoeppe

It would be good to add a few test documents to the package.

comment:24 Changed 14 months ago by git

  • Commit changed from 3acd1dd4466dcde1303982b00e55ba53bd2af2e4 to a05748b54be032ce66e25462ff297b2af7f80047

Branch pushed to git repo; I updated commit sha1. New commits:

a05748bbuild/pkgs/sage_sws2rst: More packaging

comment:25 Changed 14 months ago by mkoeppe

  • Cc kcrisman added

comment:26 in reply to: ↑ 19 ; follow-ups: Changed 14 months ago by kcrisman

How is this different from sagenb_export by the way?

This gives rst files which are quite useful for e.g. adding to Sage documentation. That was some of the main use before.

Is this basically Pablo's stuff modified for Py3? It looks like it's a required package, is that correct?

comment:27 in reply to: ↑ 26 Changed 14 months ago by mkoeppe

Replying to kcrisman:

How is this different from sagenb_export by the way?

This gives rst files which are quite useful for e.g. adding to Sage documentation. That was some of the main use before.

Thanks.

Is this basically Pablo's stuff modified for Py3?

I don't know. I just moved files around that were on this ticket already.

It looks like it's a required package, is that correct?

No, I made it an optional package, see build/pkgs/sage_sws2rst/type

comment:28 Changed 14 months ago by mkoeppe

  • Description modified (diff)

comment:29 Changed 14 months ago by mkoeppe

  • Authors set to Frédéric Chapoton, Matthias Koeppe

comment:30 in reply to: ↑ 26 Changed 14 months ago by dimpase

Replying to kcrisman:

How is this different from sagenb_export by the way?

This gives rst files which are quite useful for e.g. adding to Sage documentation. That was some of the main use before.

Jupiter's nbconvert provides conversion to rst. https://nbconvert.readthedocs.io/en/latest/ Thus, once sws is converted to jupyter, one can convert further, and the package created here is not needed.

comment:31 Changed 14 months ago by kcrisman

I suppose there is question as to whether the diagram commutes ...

comment:32 Changed 14 months ago by mkoeppe

As so often in Sage, this is a complex question - are there real users for this feature, or only imaginary users.

comment:33 Changed 14 months ago by kcrisman

At this point, unknown. I think this would be a small enough constituency if Dima's suggestion is good that we could use that. So if someone tested a few nontrivial sws worksheets and the result was usable and had roughly the same structure, I would think we could then provide this in documentation (perhaps adding something to the sagenb_export public-facing doc).

Of course, if this ticket is actually ready to go, maybe that's a moot point.

comment:34 follow-up: Changed 14 months ago by mkoeppe

The ticket is not ready to go because I have no way of testing it. If you or someone else could contribute some sws files that we can use as a little testsuite for this package, it will be easy to finish up.

comment:35 in reply to: ↑ 34 Changed 14 months ago by kcrisman

The ticket is not ready to go because I have no way of testing it. If you or someone else could contribute some sws files that we can use as a little testsuite for this package, it will be easy to finish up.

Sure thing. Trac keeps kicking me off when I try to upload attachments, so I will send them to you under separate cover - my apologies. Feel free to attach here, of course.

comment:36 Changed 14 months ago by git

  • Commit changed from a05748b54be032ce66e25462ff297b2af7f80047 to 9d976a9bd6fb3b39eb0ab462863c39860676f968

Branch pushed to git repo; I updated commit sha1. New commits:

9d976a9build/pkgs/sage_sws2rst/src: Add test with a document

comment:37 Changed 14 months ago by git

  • Commit changed from 9d976a9bd6fb3b39eb0ab462863c39860676f968 to 5e213d573107936931c5f1f83918a257cff55240

Branch pushed to git repo; I updated commit sha1. New commits:

5e213d5build/pkgs/sage_sws2rst/src/sage_sws2rst/comments2rst.py: Adjustment to bs4

comment:38 Changed 14 months ago by mkoeppe

I added one of the test worksheets from Karl-Dieter.

After sage -i tox, the code can now be tested with it using

./sage -sh -c '(cd build/pkgs/sage_sws2rst/src && tox -v -v)'

It will need adjustments for python 3 and for the beautifulsoup 3->4 upgrade. Currently it fails as follows:

  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sage_sws2rst/src/.tox/python/lib/python3.7/site-packages/sage_sws2rst/comments2rst.py", line 146, in replace_latex
    parts = single_dollar.split(unicode(t))
NameError: name 'unicode' is not defined

comment:39 Changed 14 months ago by mkoeppe

I'll stop here for now

comment:40 Changed 14 months ago by mkoeppe

Thierry, please feel free to take over from here...

comment:41 Changed 14 months ago by git

  • Commit changed from 5e213d573107936931c5f1f83918a257cff55240 to dc35ec2da534c17dca9881082f1b545bef5b212a

Branch pushed to git repo; I updated commit sha1. New commits:

f07ead8build/pkgs/sage_sws2rst/src: Run 2to3
dc35ec2build/pkgs/sage_sws2rst: Fixup

comment:42 Changed 14 months ago by mkoeppe

This version seems to work to some extent.

comment:43 Changed 14 months ago by git

  • Commit changed from dc35ec2da534c17dca9881082f1b545bef5b212a to 5262e84c49a89fd1b751b45b306c86ec4017caae

Branch pushed to git repo; I updated commit sha1. New commits:

ebe3872build/pkgs/sage_sws2rst/src/test: Add another example file
5262e84build/pkgs/sage_sws2rst/src/.gitignore: New

comment:44 Changed 14 months ago by mkoeppe

  • Status changed from needs_work to needs_review

For each processed file, there are a number of warnings Warning: node not supported (or something else?) None. I don't know if this is a problem.

comment:45 Changed 14 months ago by git

  • Commit changed from 5262e84c49a89fd1b751b45b306c86ec4017caae to 6ce42f81124d078408f875975eadfd9677be9e18

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

19cb568Fixup
5e636c7build/pkgs/sage_sws2rst: Add dependency, fix import
e29221fsrc/bin/sage: Restore --sws2rst option
132d3f1build/pkgs/sage_sws2rst: More packaging
916db99build/pkgs/sage_sws2rst/src: Add test with a document
19c541abuild/pkgs/sage_sws2rst/src/sage_sws2rst/comments2rst.py: Adjustment to bs4
61bd8b3build/pkgs/sage_sws2rst/src: Run 2to3
5b76485build/pkgs/sage_sws2rst: Fixup
96a4c8fbuild/pkgs/sage_sws2rst/src/test: Add another example file
6ce42f8build/pkgs/sage_sws2rst/src/.gitignore: New

comment:46 Changed 14 months ago by git

  • Commit changed from 6ce42f81124d078408f875975eadfd9677be9e18 to 53997700ee5e86b06d62607dca70204cc12a6a44

Branch pushed to git repo; I updated commit sha1. New commits:

5399770build/pkgs/sage_sws2rst: Update metadata

comment:48 Changed 14 months ago by mkoeppe

  • Description modified (diff)
Note: See TracTickets for help on using tickets.