Opened 3 years ago

Closed 2 years ago

#28838 closed defect (fixed)

Resurrect "sage -sws2rst" via a pip-installable package sage_sws2rst

Reported by: Thierry Monteil Owned by:
Priority: critical Milestone: sage-9.2
Component: python3 Keywords:
Cc: Volker Braun, Dima Pasechnik, Karl-Dieter Crisman, John Palmieri Merged in:
Authors: Frédéric Chapoton, Matthias Koeppe, Sébastien Labbé Reviewers: Dima Pasechnik, Sébastien Labbé, Matthias Koeppe
Report Upstream: N/A Work issues:
Branch: c026892 (Commits, GitHub, GitLab) Commit: c026892970b67a8c664530e1596890d04d364589
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

An important feature is to still be able to handle old .sws worksheets because a lot of pedagogical material was written on that format.

With the removal of sagenb, also sws2rst disappeared, see #30076 and related tickets.

We extract and translate the required material from sagenb to a new optional package sage_sws2rst that makes sage -sws2rst work.

As of this ticket, the source code of sage_sws2rst is in build/pkgs/sage_sws2rst/src, which is a pip-installable Python package; and one can build an sdist from there, for example using the spkg-src script. The source tree could as well be split out to a separate repository.

See also:

Change History (90)

comment:1 Changed 3 years ago by François Bissey

Is converting .sws file covered by sagenb_export?

comment:2 Changed 3 years ago by Thierry Monteil

Cc: Volker Braun added

It seems not directly: after a quick look, the script seems to parse the html page, but maybe it could try to parse the html that belongs to the .sws archive (not sure if it has the same structure). Let me CC Volker for a better understanding of what is doable.

comment:3 Changed 3 years ago by Frédéric Chapoton

Branch: u/chapoton/28838
Commit: 8b5de13003724601c70e36c1a11aa83db5fdaff8

Here is a first sketch. The moved files are not at the level of sage requirements, as many doctests are missing. If this matters for you, please take the job.


New commits:

8b5de13first sketch of sws2rst tools (moved from sagenb)

comment:4 Changed 3 years ago by Thierry Monteil

Shouldn't this go to a subdirectory of src/ext/ (e.g. src/ext/sws2rst/) ?

comment:5 Changed 3 years ago by Frédéric Chapoton

Salut Thierry.

Feel free to put it anywhere you like. I am not going to do anything more here.

comment:6 Changed 3 years ago by Erik Bray

Milestone: sage-9.0sage-9.1

Ticket retargeted after milestone closed

comment:7 Changed 3 years ago by Matthias Köppe

Milestone: sage-9.1sage-9.2

comment:8 Changed 2 years ago by Matthias Köppe

Cc: Dima Pasechnik added
Milestone: sage-9.2sage-duplicate/invalid/wontfix
Status: newneeds_review

outdated, sws stuff was removed in #30076

comment:9 Changed 2 years ago by Thierry Monteil

Milestone: sage-duplicate/invalid/wontfixsage-9.3
Status: needs_reviewneeds_work

It is even more important now that there is no sagenb anymore, while there are still sws files that people might want to convert (e.g. from a former course). I will try to work on this.

comment:10 Changed 2 years ago by Dima Pasechnik

Milestone: sage-9.3sage-duplicate/invalid/wontfix
Reviewers: Dima Pasechnik
Status: needs_workpositive_review

it needs sagenb, a non-starter.

comment:11 in reply to:  10 Changed 2 years ago by Thierry Monteil

Milestone: sage-duplicate/invalid/wontfixsage-9.2
Status: positive_reviewneeds_work

Replying to dimpase:

it needs sagenb, a non-starter.

It only needs a small part of it (see Frédéric's patch), and the feature is still needed.

comment:12 Changed 2 years ago by Matthias Köppe

Given that this has extra dependencies (beautifulsoup) that we have just removed, this should really be packaged as a pip-installable little package and NOT be put into sage.misc.

comment:13 Changed 2 years ago by Matthias Köppe

Summary: sage -sws2rst brokenResurrect "sage -sws2rst" via a pip-installable package sage_sws2rst

comment:14 Changed 2 years ago by Dima Pasechnik

I don't see why such an extraction cannot be carried out in a version of Sage where this tool still works.

comment:15 Changed 2 years ago by Matthias Köppe

It seems like a great idea to me to provide this converter - as long as this code is not dumped into sagelib.

comment:16 Changed 2 years ago by Matthias Köppe

I'll be happy to help with packaging

comment:17 in reply to:  9 Changed 2 years ago by Matthias Köppe

Replying to tmonteil:

It is even more important now that there is no sagenb anymore, while there are still sws files that people might want to convert (e.g. from a former course). I will try to work on this.

Any progress?

comment:18 Changed 2 years ago by Matthias Köppe

Branch: u/chapoton/28838u/mkoeppe/28838

comment:19 Changed 2 years ago by Matthias Köppe

Commit: 8b5de13003724601c70e36c1a11aa83db5fdaff8a205ec244b29b75a83581c1df9876f1a3bf824ba

How is this different from sagenb_export by the way?


New commits:

c93de0bfirst sketch of sws2rst tools (moved from sagenb)
fa9fb28Move sws2rst code to pip-installable package build/pkgs/sage_sws2rst/src
a205ec2Fixup

comment:20 Changed 2 years ago by git

Commit: a205ec244b29b75a83581c1df9876f1a3bf824ba9464457106de708fe31d51806654b47fc19a677f

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

dc41e19first sketch of sws2rst tools (moved from sagenb)
fe60540Move sws2rst code to pip-installable package build/pkgs/sage_sws2rst/src
9464457Fixup

comment:21 Changed 2 years ago by git

Commit: 9464457106de708fe31d51806654b47fc19a677f0408e6063f7c2f919fb88edacb9f9a97c550afbc

Branch pushed to git repo; I updated commit sha1. New commits:

0408e60build/pkgs/sage_sws2rst: Add dependency, fix import

comment:22 Changed 2 years ago by git

Commit: 0408e6063f7c2f919fb88edacb9f9a97c550afbc3acd1dd4466dcde1303982b00e55ba53bd2af2e4

Branch pushed to git repo; I updated commit sha1. New commits:

3acd1ddsrc/bin/sage: Restore --sws2rst option

comment:23 Changed 2 years ago by Matthias Köppe

It would be good to add a few test documents to the package.

comment:24 Changed 2 years ago by git

Commit: 3acd1dd4466dcde1303982b00e55ba53bd2af2e4a05748b54be032ce66e25462ff297b2af7f80047

Branch pushed to git repo; I updated commit sha1. New commits:

a05748bbuild/pkgs/sage_sws2rst: More packaging

comment:25 Changed 2 years ago by Matthias Köppe

Cc: Karl-Dieter Crisman added

comment:26 in reply to:  19 ; Changed 2 years ago by Karl-Dieter Crisman

How is this different from sagenb_export by the way?

This gives rst files which are quite useful for e.g. adding to Sage documentation. That was some of the main use before.

Is this basically Pablo's stuff modified for Py3? It looks like it's a required package, is that correct?

comment:27 in reply to:  26 Changed 2 years ago by Matthias Köppe

Replying to kcrisman:

How is this different from sagenb_export by the way?

This gives rst files which are quite useful for e.g. adding to Sage documentation. That was some of the main use before.

Thanks.

Is this basically Pablo's stuff modified for Py3?

I don't know. I just moved files around that were on this ticket already.

It looks like it's a required package, is that correct?

No, I made it an optional package, see build/pkgs/sage_sws2rst/type

comment:28 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:29 Changed 2 years ago by Matthias Köppe

Authors: Frédéric Chapoton, Matthias Koeppe

comment:30 in reply to:  26 Changed 2 years ago by Dima Pasechnik

Replying to kcrisman:

How is this different from sagenb_export by the way?

This gives rst files which are quite useful for e.g. adding to Sage documentation. That was some of the main use before.

Jupiter's nbconvert provides conversion to rst. https://nbconvert.readthedocs.io/en/latest/ Thus, once sws is converted to jupyter, one can convert further, and the package created here is not needed.

comment:31 Changed 2 years ago by Karl-Dieter Crisman

I suppose there is question as to whether the diagram commutes ...

comment:32 Changed 2 years ago by Matthias Köppe

As so often in Sage, this is a complex question - are there real users for this feature, or only imaginary users.

comment:33 Changed 2 years ago by Karl-Dieter Crisman

At this point, unknown. I think this would be a small enough constituency if Dima's suggestion is good that we could use that. So if someone tested a few nontrivial sws worksheets and the result was usable and had roughly the same structure, I would think we could then provide this in documentation (perhaps adding something to the sagenb_export public-facing doc).

Of course, if this ticket is actually ready to go, maybe that's a moot point.

comment:34 Changed 2 years ago by Matthias Köppe

The ticket is not ready to go because I have no way of testing it. If you or someone else could contribute some sws files that we can use as a little testsuite for this package, it will be easy to finish up.

comment:35 in reply to:  34 Changed 2 years ago by Karl-Dieter Crisman

The ticket is not ready to go because I have no way of testing it. If you or someone else could contribute some sws files that we can use as a little testsuite for this package, it will be easy to finish up.

Sure thing. Trac keeps kicking me off when I try to upload attachments, so I will send them to you under separate cover - my apologies. Feel free to attach here, of course.

comment:36 Changed 2 years ago by git

Commit: a05748b54be032ce66e25462ff297b2af7f800479d976a9bd6fb3b39eb0ab462863c39860676f968

Branch pushed to git repo; I updated commit sha1. New commits:

9d976a9build/pkgs/sage_sws2rst/src: Add test with a document

comment:37 Changed 2 years ago by git

Commit: 9d976a9bd6fb3b39eb0ab462863c39860676f9685e213d573107936931c5f1f83918a257cff55240

Branch pushed to git repo; I updated commit sha1. New commits:

5e213d5build/pkgs/sage_sws2rst/src/sage_sws2rst/comments2rst.py: Adjustment to bs4

comment:38 Changed 2 years ago by Matthias Köppe

I added one of the test worksheets from Karl-Dieter.

After sage -i tox, the code can now be tested with it using

./sage -sh -c '(cd build/pkgs/sage_sws2rst/src && tox -v -v)'

It will need adjustments for python 3 and for the beautifulsoup 3->4 upgrade. Currently it fails as follows:

  File "/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/pkgs/sage_sws2rst/src/.tox/python/lib/python3.7/site-packages/sage_sws2rst/comments2rst.py", line 146, in replace_latex
    parts = single_dollar.split(unicode(t))
NameError: name 'unicode' is not defined

comment:39 Changed 2 years ago by Matthias Köppe

I'll stop here for now

comment:40 Changed 2 years ago by Matthias Köppe

Thierry, please feel free to take over from here...

comment:41 Changed 2 years ago by git

Commit: 5e213d573107936931c5f1f83918a257cff55240dc35ec2da534c17dca9881082f1b545bef5b212a

Branch pushed to git repo; I updated commit sha1. New commits:

f07ead8build/pkgs/sage_sws2rst/src: Run 2to3
dc35ec2build/pkgs/sage_sws2rst: Fixup

comment:42 Changed 2 years ago by Matthias Köppe

This version seems to work to some extent.

comment:43 Changed 2 years ago by git

Commit: dc35ec2da534c17dca9881082f1b545bef5b212a5262e84c49a89fd1b751b45b306c86ec4017caae

Branch pushed to git repo; I updated commit sha1. New commits:

ebe3872build/pkgs/sage_sws2rst/src/test: Add another example file
5262e84build/pkgs/sage_sws2rst/src/.gitignore: New

comment:44 Changed 2 years ago by Matthias Köppe

Status: needs_workneeds_review

For each processed file, there are a number of warnings Warning: node not supported (or something else?) None. I don't know if this is a problem.

comment:45 Changed 2 years ago by git

Commit: 5262e84c49a89fd1b751b45b306c86ec4017caae6ce42f81124d078408f875975eadfd9677be9e18

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

19cb568Fixup
5e636c7build/pkgs/sage_sws2rst: Add dependency, fix import
e29221fsrc/bin/sage: Restore --sws2rst option
132d3f1build/pkgs/sage_sws2rst: More packaging
916db99build/pkgs/sage_sws2rst/src: Add test with a document
19c541abuild/pkgs/sage_sws2rst/src/sage_sws2rst/comments2rst.py: Adjustment to bs4
61bd8b3build/pkgs/sage_sws2rst/src: Run 2to3
5b76485build/pkgs/sage_sws2rst: Fixup
96a4c8fbuild/pkgs/sage_sws2rst/src/test: Add another example file
6ce42f8build/pkgs/sage_sws2rst/src/.gitignore: New

comment:46 Changed 2 years ago by git

Commit: 6ce42f81124d078408f875975eadfd9677be9e1853997700ee5e86b06d62607dca70204cc12a6a44

Branch pushed to git repo; I updated commit sha1. New commits:

5399770build/pkgs/sage_sws2rst: Update metadata

comment:48 Changed 2 years ago by Matthias Köppe

Description: modified (diff)

comment:49 Changed 2 years ago by git

Commit: 53997700ee5e86b06d62607dca70204cc12a6a44a54aecf6e8b6db7065c451bf43173aedb169a2af

Branch pushed to git repo; I updated commit sha1. New commits:

5b581a6src/doc/en/thematic_tutorials/sws2rst.rst: Restore
a54aecfsrc/doc/en/thematic_tutorials/sws2rst.rst: Update for Sage 9.2

comment:50 Changed 2 years ago by Sébastien Labbé

It has been a while I used the sagenb, but doing a find . -name *.sws on my machine made me rediscover few .sws including some available online. I copy the link below if it may be useful for testing:

http://slabbe.org/Sage/2010-perpignan/CIRM_Tutoriel_3_francais.sws
http://slabbe.org/Sage/2010-perpignan/Sage_tour_horizon.sws
http://slabbe.org/Sage/2010-perpignan/CIRM_Tutorial_3.sws
http://slabbe.org/Sage/2010-perpignan/5, 6, 100.sws
http://slabbe.org/Sage/2010-perpignan/La_Suite_Un.sws
http://slabbe.org/Sage/2010-perpignan/Outils de dessins.sws
http://slabbe.org/Sage/2010-perpignan/CIRM_Tutorial_1.sws
http://slabbe.org/Sage/2010-perpignan/CIRM_Tutoriel_1_francais.sws
http://slabbe.org/Sage/2010-days25.5-tutoriel1.sws
http://slabbe.org/Sage/2010-11-python-montreal-sage-demo.sws
http://slabbe.org/Sage/2009-canadam-sagewords.sws

I guess we can find more sws files in the wiki.sagemath.org

Last edited 2 years ago by Sébastien Labbé (previous) (diff)

comment:51 Changed 2 years ago by Sébastien Labbé

I get an error while building the doc:

[dochtml] Error building the documentation.
...
[dochtml] OSError: 
/home/slabbe/GitBox/sage/src/doc/en/thematic_tutorials/sws2rst.rst: 
WARNING: document isn't included in any toctree

... but I confirm that doc builds correctly *after* running sage -i sage_sws2rst

Last edited 2 years ago by Sébastien Labbé (previous) (diff)

comment:52 Changed 2 years ago by Dima Pasechnik

There is a typo in

--- a/src/doc/en/thematic_tutorials/index.rst
+++ b/src/doc/en/thematic_tutorials/index.rst
@@ -119,6 +119,8 @@ Advanced Programming
 Documentation
 =============
 
+* :ref:`sws2srt`
+
 .. Sage development
 .. ----------------
  • it should say sws2rst, obviously

comment:53 Changed 2 years ago by Dima Pasechnik

Status: needs_reviewneeds_work

comment:54 Changed 2 years ago by Sébastien Labbé

Doing sage -sws2rst CIRM_Tutorial_1.sws creates a rst file with still a *lot* of html stuff. Is it normal? or beautiful soup does not eat the soup well?

Also doing rst2html.py CIRM_Tutorial_1.rst gives lots of warning of the kind

CIRM_Tutorial_1.rst:83: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent.
...

because a line is missing. More precisely

.. end of output
<html><head></head><body></body></html>

should be better as:

.. end of output

<html><head></head><body></body></html>

comment:55 Changed 2 years ago by Matthias Köppe

All - please feel free to contribute to the branch

comment:56 Changed 2 years ago by Sébastien Labbé

I wrote the rst2txt and rst2sws long time ago, but sws2rst is more work (parsing html is so fun). The sws2rst script was written thanks to Pablo Angulo at the time. Let me try to spend some (limited) time on it.

Question: how does the sage -n export work? does it go through rst?

comment:57 Changed 2 years ago by Sébastien Labbé

Branch: u/mkoeppe/28838public/28838
Commit: a54aecf6e8b6db7065c451bf43173aedb169a2afdc7d8b73074b8a6c7a9107ac04bcb15f21a41ee0

adding two commits fixing two of the issues.

But there are still a lot of html in the result. Also, warnings are printed during the execution. Maybe the way we call beautiful soup needs to be improved?

$ sage -sws2rst CIRM_Tutorial_1.sws 
Processing CIRM_Tutorial_1.sws
/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage_sws2rst/comments2rst.py:396:
GuessedAtParserWarning: No parser was explicitly specified, so I'm using the 
best available HTML parser for this system ("html5lib"). This usually isn't a 
problem, but if you run this code on another system, or in a different virtual 
environment, it may use a different parser and behave differently.

The code that caused this warning is on line 396 of the file 
/home/slabbe/GitBox/sage/local/lib/python3.7/site-packages/sage_sws2rst/comments2rst.py. 
To get rid of this warning, pass the additional argument 'features="html5lib"'
to the BeautifulSoup constructor.

  soup = BeautifulSoup(text,
Warning: node not supported (or something else?) html
Warning: node not supported (or something else?) html
Warning: node not supported (or something else?) html
...
Warning: node not supported (or something else?) html
Warning: node not supported (or something else?) html
Warning: node not supported (or something else?) html
File at CIRM_Tutorial_1.rst
Image directory at CIRM_Tutorial_1_media

New commits:

2ad7f9d28838: fix typo sws2srt -> sws2rst
dc7d8b7adding an empty line after '.. end of output' to avoid warning from docutils rst2html

comment:58 Changed 2 years ago by git

Commit: dc7d8b73074b8a6c7a9107ac04bcb15f21a41ee00df2671961172e846c942b361e0db4d8d4f56d6f

Branch pushed to git repo; I updated commit sha1. New commits:

0df267128838: sagenb.misc -> sage_sws2rst

comment:59 Changed 2 years ago by git

Commit: 0df2671961172e846c942b361e0db4d8d4f56d6f691e22ce184930f8eb93884516c5c8b9426f3e2b

Branch pushed to git repo; I updated commit sha1. New commits:

691e22c28838: sagenb.misc -> sage_sws2rst

comment:60 Changed 2 years ago by git

Commit: 691e22ce184930f8eb93884516c5c8b9426f3e2bbe6307eb69c2e656bd91b87709999f161e973bd0

Branch pushed to git repo; I updated commit sha1. New commits:

be6307e28838: fix one doctest

comment:61 Changed 2 years ago by git

Commit: be6307eb69c2e656bd91b87709999f161e973bd0447d733124f73932b7a85e5dbb9f2dc18422552a

Branch pushed to git repo; I updated commit sha1. New commits:

447d73328838: from Beautiful -> from bs4

comment:62 Changed 2 years ago by Sébastien Labbé

The goal is now to make the following to work:

$ sage -t --optional=sage,beautifulsoup build/pkgs/sage_sws2rst/src/sage_sws2rst
...
-----------------------------------------------------------
sage -t --random-seed=0 build/pkgs/sage_sws2rst/src/sage_sws2rst/comments2rst.py  # 8 doctests failed
-----------------------------------------------------------

For example:

**********************************************************************
File "comments2rst.py", line 135, in sage_sws2rst.comments2rst.replace_latex
Failed example:
    s # optional - beautifulsoup
Expected:
    <p>Some <strong>latex: :math:`e^\pi i=-1`</strong></p>
Got:
    <html><head></head><body><p>Some <strong>latex:  :math:`e^\pi i=-1`</strong></p></body></html>
**********************************************************************
File "comments2rst.py", line 139, in sage_sws2rst.comments2rst.replace_latex
Failed example:
    s # optional - beautifulsoup
Expected:
    <p><strong>2\+2 \| 1\+3</strong></p>
Got:
    <html><head></head><body><p><strong>2\+2 \| 1\+3</strong></p></body></html>
**********************************************************************

comment:63 Changed 2 years ago by git

Commit: 447d733124f73932b7a85e5dbb9f2dc18422552a8f5f24d1d33e32f4a14864e17b5b3b87ad47b1a2

Branch pushed to git repo; I updated commit sha1. New commits:

8f5f24d28838: using features='html.parser' with beautifulsoup4

comment:64 Changed 2 years ago by Sébastien Labbé

$ sage -t --optional=sage,beautifulsoup build/pkgs/sage_sws2rst/src/sage_sws2rst/

now gives

Doctesting 4 files.
sage -t --random-seed=0 build/pkgs/sage_sws2rst/src/sage_sws2rst/__init__.py
    [0 tests, 0.00 s]
sage -t --random-seed=0 build/pkgs/sage_sws2rst/src/sage_sws2rst/results2rst.py
    [6 tests, 0.01 s]
sage -t --random-seed=0 build/pkgs/sage_sws2rst/src/sage_sws2rst/comments2rst.py
    [30 tests, 0.07 s]
sage -t --random-seed=0 build/pkgs/sage_sws2rst/src/sage_sws2rst/worksheet2rst.py
    [0 tests, 0.00 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------

comment:65 Changed 2 years ago by Sébastien Labbé

This now looks much better with no html code in the generated rst file. For example:

$ wget http://slabbe.org/Sage/2010-perpignan/CIRM_Tutorial_1.sws
$ sage -sws2rst CIRM_Tutorial_1.sws
Processing CIRM_Tutorial_1.sws
File at CIRM_Tutorial_1.rst
Image directory at CIRM_Tutorial_1_media
$ rst2html.py CIRM_Tutorial_1.rst CIRM_Tutorial_1.html
CIRM_Tutorial_1.rst:182: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent.
CIRM_Tutorial_1.rst:269: (WARNING/2) Inline strong start-string without end-string.
CIRM_Tutorial_1.rst:343: (WARNING/2) Inline strong start-string without end-string.
$ open CIRM_Tutorial_1.html 

The above means few ReStructuredText syntax errors are present in the generated rst file...

Also, it now looks like .. end of output is followed by more than one empty lines now. So let me remove the new lines I added few commits before.

comment:66 Changed 2 years ago by git

Commit: 8f5f24d1d33e32f4a14864e17b5b3b87ad47b1a26fa481a4b7ba57776040e7db1eaecafc4b120751

Branch pushed to git repo; I updated commit sha1. New commits:

6fa481a28838: remove empty line after '.. end of output'

comment:67 Changed 2 years ago by git

Commit: 6fa481a4b7ba57776040e7db1eaecafc4b120751f63b3c5c4832aa521e0b2f03f236b03021d32e9c

Branch pushed to git repo; I updated commit sha1. New commits:

f63b3c528838: adding a space after a end strong

comment:68 Changed 2 years ago by Sébastien Labbé

Authors: Frédéric Chapoton, Matthias KoeppeFrédéric Chapoton, Matthias Koeppe, Sébastien Labbé

Ok, so, to me, this is good enough. The remaining issues are neglectable and easy to fix by hand by the user knowing ReStructuredText syntax.

I give a positive review to all commits before mines.

Few commits I did today now need review.

Question: is sage -t --optional=sage,beautifulsoup build/pkgs/sage_sws2rst/src/sage_sws2rst/ going to be tested by the normal sage -t -a? (i'm not used with these kind of optional but included packages)

comment:69 Changed 2 years ago by Sébastien Labbé

Status: needs_workneeds_review

comment:70 Changed 2 years ago by git

Commit: f63b3c5c4832aa521e0b2f03f236b03021d32e9cfe3d62ddde9d112ba2be27949ccba8902cf6b4f1

Branch pushed to git repo; I updated commit sha1. New commits:

fe3d62dbuild/pkgs/sage_sws2rst: Convert doctests to plain python doctests, run in spkg-check

comment:71 Changed 2 years ago by Matthias Köppe

Reviewers: Dima PasechnikDima Pasechnik, Sébastien Labbé, Matthias Koeppe

comment:72 Changed 2 years ago by Matthias Köppe

The doctests now run when you install the package with SAGE_CHECK=yes

comment:73 Changed 2 years ago by Sébastien Labbé

Great!

Let me add a commit...

comment:74 Changed 2 years ago by git

Commit: fe3d62ddde9d112ba2be27949ccba8902cf6b4f1add4cc785dde2f58bb8670a853a3d5e65b4d0992

Branch pushed to git repo; I updated commit sha1. New commits:

add4cc728838: more robust handling of strong and em, added doctests

comment:75 Changed 2 years ago by Sébastien Labbé

I am trying to run the tests with SAGE_CHECK, but before that I now get

$ make sage_sws2rst
...
[sage_sws2rst-none] /home/slabbe/GitBox/sage/build/pkgs/sage_sws2rst/spkg-install: 6: source: not found
[sage_sws2rst-none] Error: failed to source /home/slabbe/GitBox/sage/build/bin/sage-dist-helpers
[sage_sws2rst-none] Is /home/slabbe/GitBox/sage the correct SAGE_ROOT?
...
***************************************************************
Error building Sage.

The following package(s) may have failed to build (not necessarily
during this run of 'make sage_sws2rst'):

* package:         sage_sws2rst-none
  last build time: sept. 6 22:09
  log file:        /home/slabbe/GitBox/sage/logs/pkgs/sage_sws2rst-none.log
...

Same thing if I run SAGE_CHECK=yes make sage_sws2rst. Why does it failed to source?

Last edited 2 years ago by Sébastien Labbé (previous) (diff)

comment:76 Changed 2 years ago by Matthias Köppe

Ah, sorry, I'll fix that

comment:77 Changed 2 years ago by Sébastien Labbé

So if I undo the change

-#! /usr/bin/env bash
+#! /bin/sh

then I am able to make sage_sws2rst. But then SAGE_CHECK=yes make sage_sws2rst fails with

[sage_sws2rst-none] Successfully installed sage-sws2rst-9.2
[sage_sws2rst-none] Removed build tracker: '/tmp/pip-req-tracker-jp5og0mz'
[sage_sws2rst-none] /home/slabbe/GitBox/sage/build/pkgs/sage_sws2rst/spkg-install: 
ligne 17: ./spkg-check: Aucun fichier ou dossier de ce type

Indeed, where is the spkg-check file?

comment:78 Changed 2 years ago by git

Commit: add4cc785dde2f58bb8670a853a3d5e65b4d09922e630951d379471b34a3865332648605b4bca2f4

Branch pushed to git repo; I updated commit sha1. New commits:

2e63095build/pkgs/sage_sws2rst/spkg-install: Remove bashism

comment:79 Changed 2 years ago by git

Commit: 2e630951d379471b34a3865332648605b4bca2f4a3630739a056769814ed7f4aeb26ee898aa4046a

Branch pushed to git repo; I updated commit sha1. New commits:

a363073build/pkgs/sage_sws2rst/spkg-check: New

comment:80 Changed 2 years ago by Sébastien Labbé

make sage_sws2rst now gives:

[sage_sws2rst-none] /home/slabbe/GitBox/sage/build/pkgs/sage_sws2rst/spkg-install: 
239: /home/slabbe/GitBox/sage/build/bin/sage-dist-helpers: 
Syntax error: "(" unexpected (expecting "}")

comment:81 Changed 2 years ago by git

Commit: a3630739a056769814ed7f4aeb26ee898aa4046a8e277340e274084d130f28ff51926ba6e9aa555e

Branch pushed to git repo; I updated commit sha1. New commits:

8e27734build/pkgs/sage_sws2rst/spkg-install: Back to using bash

comment:82 Changed 2 years ago by Sébastien Labbé

Good. make sage_sws2rst works and SAGE_CHECK=yes make sage_sws2rst now gives:

[sage_sws2rst-none] Successfully built sage-sws2rst
[sage_sws2rst-none] Installing collected packages: sage-sws2rst
[sage_sws2rst-none]   Created temporary directory: /tmp/pip-unpacked-wheel-jwcrc2ua
[sage_sws2rst-none] 
[sage_sws2rst-none] Successfully installed sage-sws2rst-9.2
[sage_sws2rst-none] Removed build tracker: '/tmp/pip-req-tracker-w5ksuaxe'
[sage_sws2rst-none] + set -e
[sage_sws2rst-none] + sage-sws2rst -h
[sage_sws2rst-none] + python3 sage_sws2rst/comments2rst.py
[sage_sws2rst-none] + python3 sage_sws2rst/results2rst.py
[sage_sws2rst-none] + cd test
[sage_sws2rst-none] + sage-sws2rst Adding_Pictures_and_screenshots.sws
[sage_sws2rst-none] Processing Adding_Pictures_and_screenshots.sws
[sage_sws2rst-none] File at Adding_Pictures_and_screenshots.rst
[sage_sws2rst-none] Image directory at Adding_Pictures_and_screenshots_media
[sage_sws2rst-none] + sage-sws2rst MAT_141_day_25.sws
[sage_sws2rst-none] Processing MAT_141_day_25.sws
[sage_sws2rst-none] File at MAT_141_day_25.rst
[sage_sws2rst-none] Image directory at MAT_141_day_25_media
touch "/home/slabbe/GitBox/sage/local/var/lib/sage/installed/sage_sws2rst-none"
make[1] : on quitte le répertoire « /home/slabbe/GitBox/sage/build/make »

real	0m3,936s
user	0m3,554s
sys	0m0,403s
Sage build/upgrade complete!

Seems good!

comment:83 Changed 2 years ago by Sébastien Labbé

In a folder with the following makefile:

download:
	wget http://slabbe.org/Sage/2010-perpignan/CIRM_Tutoriel_3_francais.sws
	wget http://slabbe.org/Sage/2010-perpignan/Sage_tour_horizon.sws
	wget http://slabbe.org/Sage/2010-perpignan/CIRM_Tutorial_3.sws
	wget "http://slabbe.org/Sage/2010-perpignan/5, 6, 100.sws"
	wget http://slabbe.org/Sage/2010-perpignan/La_Suite_Un.sws
	wget "http://slabbe.org/Sage/2010-perpignan/Outils de dessins.sws"
	wget http://slabbe.org/Sage/2010-perpignan/CIRM_Tutorial_1.sws
	wget http://slabbe.org/Sage/2010-perpignan/CIRM_Tutoriel_1_francais.sws
	wget http://slabbe.org/Sage/2010-days25.5-tutoriel1.sws
	wget http://slabbe.org/Sage/2010-11-python-montreal-sage-demo.sws
	wget http://slabbe.org/Sage/2009-canadam-sagewords.sws

sws2rst:
	sage -sws2rst CIRM_Tutoriel_3_francais.sws
	sage -sws2rst Sage_tour_horizon.sws
	sage -sws2rst CIRM_Tutorial_3.sws
	sage -sws2rst "5, 6, 100.sws"
	sage -sws2rst La_Suite_Un.sws
	sage -sws2rst "Outils de dessins.sws"
	sage -sws2rst CIRM_Tutorial_1.sws
	sage -sws2rst CIRM_Tutoriel_1_francais.sws
	sage -sws2rst 2010-days25.5-tutoriel1.sws
	sage -sws2rst 2010-11-python-montreal-sage-demo.sws
	#sage -sws2rst 2009-canadam-sagewords.sws

rst2html:
	rst2html.py CIRM_Tutoriel_3_francais.rst a.out
	rst2html.py Sage_tour_horizon.rst a.out
	rst2html.py CIRM_Tutorial_3.rst a.out
	rst2html.py 5,_6,_100.rst a.out
	rst2html.py La_Suite_Un.rst a.out
	rst2html.py Outils_de_dessins.rst a.out
	rst2html.py CIRM_Tutorial_1.rst a.out
	rst2html.py CIRM_Tutoriel_1_francais.rst a.out
	rst2html.py 2010-days25.5-tutoriel1.rst a.out
	rst2html.py 2010-11-python-montreal-sage-demo.rst a.out
	#rst2html.py 2009-canadam-sagewords.rst a.out

then make download && make sws2rst works and

make rst2html

gives only very few warnings:

$ make rst2html 
rst2html.py CIRM_Tutoriel_3_francais.rst a.out
CIRM_Tutoriel_3_francais.rst:781: (WARNING/2) Inline emphasis start-string without end-string.
rst2html.py Sage_tour_horizon.rst a.out
rst2html.py CIRM_Tutorial_3.rst a.out
CIRM_Tutorial_3.rst:754: (WARNING/2) Inline emphasis start-string without end-string.
rst2html.py 5,_6,_100.rst a.out
rst2html.py La_Suite_Un.rst a.out
rst2html.py Outils_de_dessins.rst a.out
rst2html.py CIRM_Tutorial_1.rst a.out
CIRM_Tutorial_1.rst:176: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent.
CIRM_Tutorial_1.rst:334: (WARNING/2) Inline strong start-string without end-string.
rst2html.py CIRM_Tutoriel_1_francais.rst a.out
CIRM_Tutoriel_1_francais.rst:178: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent.
CIRM_Tutoriel_1_francais.rst:338: (WARNING/2) Inline strong start-string without end-string.
rst2html.py 2010-days25.5-tutoriel1.rst a.out
2010-days25.5-tutoriel1.rst:149: (WARNING/2) Block quote ends without a blank line; unexpected unindent.
2010-days25.5-tutoriel1.rst:182: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent.
2010-days25.5-tutoriel1.rst:312: (WARNING/2) Inline strong start-string without end-string.
* Unknown equation environment pmatrix
rst2html.py 2010-11-python-montreal-sage-demo.rst a.out
2010-11-python-montreal-sage-demo.rst:495: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent.
2010-11-python-montreal-sage-demo.rst:534: (WARNING/2) Explicit markup ends without a blank line; unexpected unindent.
#rst2html.py 2009-canadam-sagewords.rst a.out

It is true, that sometimes it does not translates the html properly. For instance, the following paragraph appears in the file CIRM_Tutorial_3.rst:

`Project Euler Problem 6 <http://projecteuler.net/index.php?section=problems&id=6>`_
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

<blockquote><p>The sum of the squares of the first ten natural numbers is,</p><div style="text-align: center;">1<img alt="^(" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/><sup>2</sup><img alt=")" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/> \+ 2<img alt="^(" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/><sup>2</sup><img alt=")" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/> \+ ... \+ 10<img alt="^(" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/><sup>2</sup><img alt=")" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/> = 385</div><p>The square of the sum of the first ten natural numbers is,</p><div style="text-align: center;">(1 \+ 2 \+ ... \+ 10)<img alt="^(" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/><sup>2</sup><img alt=")" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/> = 55<img alt="^(" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/><sup>2</sup><img alt=")" src="http://projecteuler.net/index.php?section=problems&id=6" style="display: none;"/> = 3025</div><p>Hence the difference between the sum of the squares of the first ten natural numbers and the square of the sum is 3025 <img alt="−" border="0" height="3" src="http://projecteuler.net/images/symbol_minus.gif" style="vertical-align: middle;" width="9"/> 385 = 2640.</p><p>Find the difference between the sum of the squares of the first one hundred natural numbers and the square of the sum.</p></blockquote>

but I am quite sure it was the same problem before. From me it is a positive review.

Let's deal with blockquote and other missing styles in another ticket if somebody finds it desirable.

comment:84 Changed 2 years ago by Matthias Köppe

Status: needs_reviewpositive_review

comment:85 Changed 2 years ago by Volker Braun

Status: positive_reviewneeds_work
[dochtml] [thematic_] The HTML pages are in local/share/doc/sage/html/en/thematic_tutorials.
[dochtml] Error building the documentation.
[dochtml] Traceback (most recent call last):
[dochtml]   File "/home/release/Sage/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
[dochtml]     return _run_code(code, main_globals, None,
[dochtml]   File "/home/release/Sage/local/lib/python3.8/runpy.py", line 87, in _run_code
[dochtml]     exec(code, run_globals)
[dochtml]   File "/home/release/Sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__main__.py", line 2, in <module>
[dochtml]     main()
[dochtml]   File "/home/release/Sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 1730, in main
[dochtml]     builder()
[dochtml]   File "/home/release/Sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 353, in _wrapper
[dochtml]     build_many(build_other_doc, L)
[dochtml]   File "/home/release/Sage/local/lib/python3.8/site-packages/sage_setup/docbuild/__init__.py", line 296, in build_many
[dochtml]     _build_many(target, args, processes=NUM_THREADS)
[dochtml]   File "/home/release/Sage/local/lib/python3.8/site-packages/sage_setup/docbuild/utils.py", line 289, in build_many
[dochtml]     raise worker_exc.original_exception
[dochtml] OSError: /home/release/Sage/src/doc/en/thematic_tutorials/sws2rst.rst: WARNING: document isn't included in any toctree

comment:86 Changed 2 years ago by Matthias Köppe

Cc: John Palmieri added

comment:87 Changed 2 years ago by git

Commit: 8e277340e274084d130f28ff51926ba6e9aa555ec026892970b67a8c664530e1596890d04d364589

Branch pushed to git repo; I updated commit sha1. New commits:

0161466Merge tag '9.2.beta12' into t/28838/28838
c026892src/doc/en/thematic_tutorials/toctree.rst: Add sws2rst

comment:88 Changed 2 years ago by Matthias Köppe

Status: needs_workneeds_review

comment:89 Changed 2 years ago by Matthias Köppe

Status: needs_reviewpositive_review

Docbuild is fixed, back to positive review

comment:90 Changed 2 years ago by Volker Braun

Branch: public/28838c026892970b67a8c664530e1596890d04d364589
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.