Opened 7 years ago

Closed 7 years ago

Last modified 4 years ago

#13143 closed enhancement (fixed)

Use MathJax by default when building docs from Makefile

Reported by: jhpalmieri Owned by: GeorgSWeber
Priority: minor Milestone: sage-5.5
Component: build Keywords: Makefile doc MathJax sd40
Cc: nthiery Merged in: sage-5.5.beta2
Authors: John Palmieri Reviewers: Dmitrii Pasechnik
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: #9774, #13121 Stopgaps:

Description (last modified by chapoton)

At the moment, running make runs make doc, which in turn runs make doc-html. This runs Sphinx without the -j option, so MathJax (or jsMath, pre #9774) is not used: png image files are used instead. Using png images is much slower, so we should use MathJax (or jsMath) by default.

Timing and disk space information:

Tutorial without MathJax:

$ rm -rf devel/sage/doc/output
$ time sage --docbuild tutorial html
...
real 2m17.950s
user 1m29.441s
sys 0m15.478s
$ du -s -h devel/sage/doc/output
3.9M output/

With MathJax:

$ rm -rf devel/sage/doc/output
$ time sage --docbuild tutorial html -j
...
real 0m9.871s
user 0m6.530s
sys 0m1.739s
$ du -s -h devel/sage/doc/output
 8.5M output/

When building all of the documentation, the disk space actually drops, going from about 320M (without MathJax) to about 295M (with).


This change was discussed on sage-devel.

Attachments (5)

trac_13143-use-mathjax.patch (1.1 KB) - added by jhpalmieri 7 years ago.
root repo
trac_13143-root.v2.patch (1.9 KB) - added by jhpalmieri 7 years ago.
root repo
trac_13143-sage.v2.patch (4.4 KB) - added by jhpalmieri 7 years ago.
Sage library
trac_13143-optional.patch (2.2 KB) - added by jhpalmieri 7 years ago.
optional patch, independent of the others
trac_13143-pictures.patch (18.5 KB) - added by jdemeyer 7 years ago.
Sage library

Download all attachments as: .zip

Change History (48)

Changed 7 years ago by jhpalmieri

root repo

comment:1 Changed 7 years ago by jhpalmieri

  • Status changed from new to needs_review

comment:2 Changed 7 years ago by jhpalmieri

  • Description modified (diff)

comment:3 Changed 7 years ago by jhpalmieri

  • Description modified (diff)

comment:4 Changed 7 years ago by kcrisman

Here's a question. I've noticed that sometimes looking at MathJax stuff when I'm not on the internet is less than great. What is the story then?

But this certainly looks right.

Maybe something should then also be added to the developer or installation guide (wherever the make options are documented other than in the Makefile) that points out how not to use MathJax in a build. Like to do make build doc-html or whatever the correct syntax would be.

comment:5 Changed 7 years ago by jhpalmieri

  • Cc nthiery added
  • Description modified (diff)

comment:6 Changed 7 years ago by jhpalmieri

The "v2" patches do the following:

  • turn MathJax on by default, by modifying the script sage-env and by modifying how the variable's value is checked in the doc building files
  • pass the "no-pdf-links" option only when building the website document, so its presence (or lack) on the command-line doesn't cause the reference manual to be rebuilt.

comment:7 Changed 7 years ago by jhpalmieri

In response to Karl-Dieter's comment, I've added some documentation, to the "environment variables" section of the installation guide.

comment:8 Changed 7 years ago by kcrisman

Ok, this is more minimalist than I would like (i.e., have all the make targets documented somewhere) but I think it's appropriate for this ticket. I'm sorry that I don't have time to give this full review, though again it seems ok.

comment:9 Changed 7 years ago by jhpalmieri

See #13219 for documentation of the make targets.

comment:10 Changed 7 years ago by jhpalmieri

  • Dependencies changed from #9774 to #9774, #13121

comment:11 Changed 7 years ago by jhpalmieri

  • Keywords sd40 added

comment:12 Changed 7 years ago by drkirkby

I had a problem with Latex creating thousands of errors on CentOS 4.7 (an old version of Linux, which I need to run for some commercial software). See discussion at:

http://groups.google.com/forum/?fromgroups#!topic/sage-devel/7erAQRbHYUU

At John Palmieri's suggestion, I run:

$ rm -rf SAGE_ROOT/devel/sage/doc/output
$ sage --docbuild all html -j

and the documentation built OK, with no errors.

Dave

comment:13 Changed 7 years ago by jhpalmieri

  • Dependencies changed from #9774, #13121 to #9774, #13121, #6495

comment:14 Changed 7 years ago by jhpalmieri

  • Description modified (diff)

comment:15 Changed 7 years ago by jhpalmieri

  • Dependencies changed from #9774, #13121, #6495 to #9774, #13121
  • Description modified (diff)

comment:16 Changed 7 years ago by jhpalmieri

  • Description modified (diff)

Changed 7 years ago by jhpalmieri

root repo

Changed 7 years ago by jhpalmieri

Sage library

comment:17 Changed 7 years ago by hivert

Hi John,

As a prerequisite of #6495 I'm starting to review this one.

Florent

comment:18 follow-up: Changed 7 years ago by jhpalmieri

Hi Florent,

Great. Please let me know if you have any questions or comments.

John

comment:19 in reply to: ↑ 18 ; follow-ups: Changed 7 years ago by hivert

Hi John,

Replying to jhpalmieri:

Great. Please let me know if you have any questions or comments.

I'm running into some problems. Here are what I did: on a fresh 5.4.rc0, I made a clone and imported the two v2 patches. Then I removed the doc/output directory and issued a

sage -docbuild reference html

After that some pages are perfectly Ok but for some other (I can't find a pattern) there is a line in the bottom saying

Loading [MathJax]/fonts/HTML-CSS/TeX/png/imagedata.js

And after somehow ten seconds the text changes to

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

Indeed the file doesn't exists on my filesystem. By the way, I'm using Firefox 15.0 on openSuSE 12.1 64 bits.

Am I missing something trivial ?

Cheers,

Florent

comment:20 in reply to: ↑ 19 Changed 7 years ago by hivert

Sorry for the two answers.

And after somehow ten seconds the text changes to

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

You probably guessed it but after that the maths are replaced by a red [Math Processing Error].

Cheers,

Florent

comment:21 follow-up: Changed 7 years ago by jhpalmieri

Hi Florent,

I have no idea. I tried on my machine (OS X). I browsed through 10-20 pages of the reference manual and saw no problems. Does this depend on this ticket? If you don't apply the patches and run

sage -docbuild reference html -j

do you see the same problem?

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

I don't even see a "png" directory. I see "eot", "otf", and "svg" directories.

comment:22 in reply to: ↑ 21 Changed 7 years ago by hivert

Replying to jhpalmieri:

Hi Florent,

I have no idea. I tried on my machine (OS X). I browsed through 10-20 pages of the reference manual and saw no problems. Does this depend on this ticket? If you don't apply the patches and run

sage -docbuild reference html -j

do you see the same problem?

Running... I can't wait having #6495 ;-)

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

I don't even see a "png" directory. I see "eot", "otf", and "svg" directories.

Sorry I wasn't precise enough. The same holds for me.

comment:23 Changed 7 years ago by hivert

Apparently it has nothing to do with the patch... I'm trying with sage-5.3...

Florent

Last edited 7 years ago by hivert (previous) (diff)

comment:24 Changed 7 years ago by hivert

So the problem has nothing to do with the patch here. MathJax? math seems to be Ok on my laptop with sage 5.3 but are both broken with sage 5.4.rc0 and 5.4.rc1. More precisely I just did the following:

  • Build from sources a fresh 5.4.rc1
  • removed devel/sage-main/doc/output/html/en/reference
  • issued ./sage -docbuild reference html --mathjax

then the file index.html displayed correctly in firefox but devel/sage-main/doc/output/html/en/reference/sage/modular/modsym/p1list.html was broken. The error message is the same as before. Any idea ? May this be related to #9774 ?

Florent

comment:25 Changed 7 years ago by hivert

There is also a problem with compiling the doc with MathJax. Some pages use advanced LateX stuff such as pictures. Unfortunately they doesn't work with MathJax activated. See eg:

sage/graphs/base/static_sparse_graph.pyx

Cheers,

Florent

comment:26 Changed 7 years ago by jhpalmieri

Ideally, this would be dealt with by Sphinx (and I'm glad you wrote to sphinx-devel about it). But the documentation should be written so that it looks good with or without MathJax (or jsMath, in older versions of Sage). The picture mode LaTeX code isn't processed correctly with these: it only works with the pngmath Sphinx extension. (I checked some older versions of Sage, and it didn't work with jsMath either.) So I think the documentation should be changed: for pictures like these, someone should just produce some pngs and then include them, as is done in several other places in the docs. Indeed, that documentation should never have been allowed in Sage in the first place, since it didn't build using Sphinx's jsMath extension.

I've been playing around with Sphinx, trying to see if there is a way to turn extensions like pngmath on or off using a function process_mathjax called like

    app.connect('autodoc-process-docstring', process_mathjax)

in the setup function at the end of devel/sage/doc/common/conf.py, but I haven't gotten anywhere yet.

Last edited 7 years ago by jhpalmieri (previous) (diff)

comment:27 Changed 7 years ago by jhpalmieri

  • Description modified (diff)

Here is a patch which replaces several picture environments with pngs. I tried searching the Sage library for possible problems, but I don't know exactly what MathJax cannot handle, so I searched for files containing both ".. math" and "begin{". These were the only instances I found which didn't render correctly with MathJax. I'm afraid it would be almost impossible to search the whole reference manual to look for other problems, since I think they only appear upon rendering in the web browser.

comment:28 Changed 7 years ago by jhpalmieri

If this approach (pngs instead of picture environments) looks okay, I'll rebase #6495 on top of it.

comment:29 in reply to: ↑ 19 Changed 7 years ago by jhpalmieri

Replying to hivert:

Hi John,

Replying to jhpalmieri:

Great. Please let me know if you have any questions or comments.

I'm running into some problems. Here are what I did: on a fresh 5.4.rc0, I made a clone and imported the two v2 patches. Then I removed the doc/output directory and issued a

sage -docbuild reference html

After that some pages are perfectly Ok but for some other (I can't find a pattern) there is a line in the bottom saying

Loading [MathJax]/fonts/HTML-CSS/TeX/png/imagedata.js

And after somehow ten seconds the text changes to

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

Indeed the file doesn't exists on my filesystem. By the way, I'm using Firefox 15.0 on openSuSE 12.1 64 bits.

I still don't know what's going on with this. On several systems (OS X and sage.math.washington.edu), I don't see this. A few weeks ago, I made a version of the documentation using MathJax and #6495: http://sage.math.washington.edu/home/palmieri/misc/6495-jsmath/html/en/index.html. I skimmed through some pages, and don't see any of these problems. Do you have any ideas what could be causing it? I can't imagine why it would make a difference, but did you run sage -b in addition to removing doc/output? Is it repeatable, and are the web pages on which the problems occur repeatable?

comment:30 follow-up: Changed 7 years ago by jhpalmieri

As far as other possibly incompatible pieces of LaTeX in the Sage library, I haven't found any. I ran

sage: search_src('\\\\[a-zA-Z]', interact=False)

to get a list of all strings starting with a backslash and then a letter, then turned this into a list of all of the potential LaTeX commands used in Sage, and compared it to MathJax's list. For items on Sage's list that aren't recognized by MathJax, I looked at how they were used in the Sage library. This wasn't a very long list, and I admit that I wasn't extremely careful, but I didn't find any problems.

comment:31 Changed 7 years ago by jhpalmieri

(I just fixed a typo in the commit message.)

comment:32 in reply to: ↑ 30 ; follow-up: Changed 7 years ago by dimpase

  • Status changed from needs_review to needs_work

Replying to jhpalmieri:

As far as other possibly incompatible pieces of LaTeX in the Sage library, I haven't found any. I ran

sage: search_src('\\\\[a-zA-Z]', interact=False)

to get a list of all strings starting with a backslash and then a letter, then turned this into a list of all of the potential LaTeX commands used in Sage, and compared it to MathJax's list. For items on Sage's list that aren't recognized by MathJax, I looked at how they were used in the Sage library. This wasn't a very long list, and I admit that I wasn't extremely careful, but I didn't find any problems.

Here is one thing I found: compare http://www.sagemath.org/doc/thematic_tutorials/numtheory_rsa.html and the corresponding page in your output, somewhere in the middle:

Formally, let

Σ={A,B,C,...,Z}

be the set of capital letters of the English alphabet. Furthermore, let

On your pages (and also on my own build with this patch), the formula \Sigma={A,B,C,...,Z} is broken (viewed on OSX10.6.8, with Chrome or with Safari). Namely, it looks like Σ={\textttA,\textttB,\textttC,...,\textttZ}, with \texttt typeset in red.

In the source I see

.. MATH::

    \Sigma
    =
    \{ \texttt{A}, \texttt{B}, \texttt{C}, \dots, \texttt{Z} \}

which look like 100% kosher TeX to me.

Can you fix this in MathJax config?

comment:33 in reply to: ↑ 32 Changed 7 years ago by dimpase

  • Status changed from needs_work to needs_info

Replying to dimpase:

Can you fix this in MathJax config?

This appears to be a mathjax bug, of sorts, according to google. If I put the following into Macro: section of mathjax_sage.js in output/html/en/thematic_tutorials/_static/ then it displays just right (textfs and emph are added just for good measure, as they suffer the same fate, apparently).

texttt:             ['\\mathord{\\tt{\\text{#1}}}',1],
textsf:             ['\\mathord{\\sf{\\text{#1}}}',1],
emph:               ['\\textit{#1}',1]

apparently this should be added to the appropriate mathjax_sage.js_t file, although how to rebuild the docs to take care of this change this way, I don't know. If someone tells me this, I could come up with a patch.

comment:34 Changed 7 years ago by jhpalmieri

\texttt is supposed to be used in text mode, not math mode, so whether it's a MathJax bug, we should replace it with \mathtt. That fixes the problem for me. I'll add this patch in a few minutes. I also don't know what version of MathJax we have; at some point we should probably upgrade.

comment:35 Changed 7 years ago by jhpalmieri

  • Status changed from needs_info to needs_review

In fact, I think that MathJax is only designed to handle math mode, so we shouldn't expect \texttt or \textsf to work, or \emph in text mode. (Sphinx won't even call MathJax on text not between single backquotes, i.e., on non-math.) Using \emph in math mode is not very common, so I suggest not worrying about it.

comment:36 follow-up: Changed 7 years ago by jhpalmieri

Note that \texttt appears in two more places in the Sage library, but not in the documentation: in combinat/posets/linear_extension.py and in modules/vector_space_morphism.py. In the first of these, it should be replace by \mathtt, and in the second, probably by \text. These changes are not related to this ticket, but we can add them on anyway...

comment:37 in reply to: ↑ 36 Changed 7 years ago by dimpase

Replying to jhpalmieri:

Note that \texttt appears in two more places in the Sage library, but not in the documentation: in combinat/posets/linear_extension.py and in modules/vector_space_morphism.py. In the first of these, it should be replace by \mathtt, and in the second, probably by \text. These changes are not related to this ticket, but we can add them on anyway...

Right, let's do this, and hopefully be done with! Indeed, I didn't think straight about \texttt{}.

Changed 7 years ago by jhpalmieri

optional patch, independent of the others

comment:38 follow-up: Changed 7 years ago by jhpalmieri

If you think we should apply the "optional" patch, please add it to the ticket description (under the list of patches to be applied).

comment:39 in reply to: ↑ 38 Changed 7 years ago by dimpase

Replying to jhpalmieri:

If you think we should apply the "optional" patch, please add it to the ticket description (under the list of patches to be applied).

In both cases it's inside the docs for _latex function, which do not appear in the HTML documentation at all, for some reason. I guess sphynx skips stuff beginning from _. I don't know whether it's a bug or a feature.

comment:40 Changed 7 years ago by dimpase

  • Status changed from needs_review to positive_review

comment:41 Changed 7 years ago by jdemeyer

  • Milestone changed from sage-5.4 to sage-5.5
  • Reviewers set to Dmitrii Pasechnik

Changed 7 years ago by jdemeyer

Sage library

comment:42 Changed 7 years ago by jdemeyer

  • Merged in set to sage-5.5.beta2
  • Resolution set to fixed
  • Status changed from positive_review to closed

Rebased to sage-5.5.beta1.

comment:43 Changed 4 years ago by chapoton

  • Description modified (diff)
Note: See TracTickets for help on using tickets.