Use MathJax by default when building docs from Makefile

[jhpalmieri]

At the moment, running make runs make doc, which in turn runs make doc-html. This runs Sphinx without the -j option, so MathJax (or jsMath, pre #9774) is not used: png image files are used instead. Using png images is much slower, so we should use MathJax (or jsMath) by default.

Timing and disk space information:

Tutorial without MathJax:

$ rm -rf devel/sage/doc/output
$ time sage --docbuild tutorial html
...
real 2m17.950s
user 1m29.441s
sys 0m15.478s
$ du -s -h devel/sage/doc/output
3.9M output/

With MathJax:

$ rm -rf devel/sage/doc/output
$ time sage --docbuild tutorial html -j
...
real 0m9.871s
user 0m6.530s
sys 0m1.739s
$ du -s -h devel/sage/doc/output
 8.5M output/

When building all of the documentation, the disk space actually drops, going from about 320M (without MathJax) to about 295M (with).

---

• Apply trac_13143-root.v2.patch​ to the root repo.
• Apply trac_13143-sage.v2.patch​ and trac_13143-pictures.patch​ to the Sage library.

This change was discussed ​on sage-devel.

[jhpalmieri]

root repo

[kcrisman]

Here's a question. I've noticed that sometimes looking at MathJax stuff when I'm not on the internet is less than great. What is the story then?

But this certainly looks right.

Maybe something should then also be added to the developer or installation guide (wherever the make options are documented other than in the Makefile) that points out how not to use MathJax in a build. Like to do make build doc-html or whatever the correct syntax would be.

[jhpalmieri]

The "v2" patches do the following:

• turn MathJax on by default, by modifying the script sage-env and by modifying how the variable's value is checked in the doc building files
• pass the "no-pdf-links" option only when building the website document, so its presence (or lack) on the command-line doesn't cause the reference manual to be rebuilt.

[jhpalmieri]

In response to Karl-Dieter's comment, I've added some documentation, to the "environment variables" section of the installation guide.

[kcrisman]

Ok, this is more minimalist than I would like (i.e., have all the make targets documented somewhere) but I think it's appropriate for this ticket. I'm sorry that I don't have time to give this full review, though again it seems ok.

[jhpalmieri]

See #13219 for documentation of the make targets.

[drkirkby]

I had a problem with Latex creating thousands of errors on CentOS 4.7 (an old version of Linux, which I need to run for some commercial software). See discussion at:

​http://groups.google.com/forum/?fromgroups#!topic/sage-devel/7erAQRbHYUU

At John Palmieri's suggestion, I run:

$ rm -rf SAGE_ROOT/devel/sage/doc/output
$ sage --docbuild all html -j

and the documentation built OK, with no errors.

Dave

[jhpalmieri]

root repo

[jhpalmieri]

Sage library

[hivert]

Hi John,

As a prerequisite of #6495 I'm starting to review this one.

Florent

[jhpalmieri]

Hi Florent,

Great. Please let me know if you have any questions or comments.

John

[hivert]

Hi John,

Replying to jhpalmieri:

Great. Please let me know if you have any questions or comments.

I'm running into some problems. Here are what I did: on a fresh 5.4.rc0, I made a clone and imported the two v2 patches. Then I removed the doc/output directory and issued a

sage -docbuild reference html

After that some pages are perfectly Ok but for some other (I can't find a pattern) there is a line in the bottom saying

Loading [MathJax]/fonts/HTML-CSS/TeX/png/imagedata.js

And after somehow ten seconds the text changes to

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

Indeed the file doesn't exists on my filesystem. By the way, I'm using Firefox 15.0 on openSuSE 12.1 64 bits.

Am I missing something trivial ?

Cheers,

Florent

[hivert]

Sorry for the two answers.

And after somehow ten seconds the text changes to

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

You probably guessed it but after that the maths are replaced by a red [Math Processing Error].

Cheers,

Florent

[jhpalmieri]

Hi Florent,

I have no idea. I tried on my machine (OS X). I browsed through 10-20 pages of the reference manual and saw no problems. Does this depend on this ticket? If you don't apply the patches and run

sage -docbuild reference html -j

do you see the same problem?

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

I don't even see a "png" directory. I see "eot", "otf", and "svg" directories.

[hivert]

Replying to jhpalmieri:

Hi Florent,

I have no idea. I tried on my machine (OS X). I browsed through 10-20 pages of the reference manual and saw no problems. Does this depend on this ticket? If you don't apply the patches and run

sage -docbuild reference html -j

do you see the same problem?

Running... I can't wait having #6495 ;-)

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

I don't even see a "png" directory. I see "eot", "otf", and "svg" directories.

Sorry I wasn't precise enough. The same holds for me.

[hivert]

Apparently it has nothing to do with the patch... I'm trying with sage-5.3...

Florent

[hivert]

So the problem has nothing to do with the patch here. MathJax? math seems to be Ok on my laptop with sage 5.3 but are both broken with sage 5.4.rc0 and 5.4.rc1. More precisely I just did the following:

• Build from sources a fresh 5.4.rc1
• removed devel/sage-main/doc/output/html/en/reference
• issued ./sage -docbuild reference html --mathjax

then the file index.html displayed correctly in firefox but devel/sage-main/doc/output/html/en/reference/sage/modular/modsym/p1list.html was broken. The error message is the same as before. Any idea ? May this be related to #9774 ?

Florent

[hivert]

There is also a problem with compiling the doc with MathJax. Some pages use advanced LateX stuff such as pictures. Unfortunately they doesn't work with MathJax activated. See eg:

sage/graphs/base/static_sparse_graph.pyx

Cheers,

Florent

[jhpalmieri]

Ideally, this would be dealt with by Sphinx (and I'm glad you wrote to sphinx-devel about it). But the documentation should be written so that it looks good with or without MathJax (or jsMath, in older versions of Sage). The picture mode LaTeX code isn't processed correctly with these: it only works with the pngmath Sphinx extension. (I checked some older versions of Sage, and it didn't work with jsMath either.) So I think the documentation should be changed: for pictures like these, someone should just produce some pngs and then include them, as is done in several other places in the docs. Indeed, that documentation should never have been allowed in Sage in the first place, since it didn't build using Sphinx's jsMath extension.

I've been playing around with Sphinx, trying to see if there is a way to turn extensions like pngmath on or off using a function process_mathjax called like

    app.connect('autodoc-process-docstring', process_mathjax)

in the setup function at the end of devel/sage/doc/common/conf.py, but I haven't gotten anywhere yet.

[jhpalmieri]

Here is a patch which replaces several picture environments with pngs. I tried searching the Sage library for possible problems, but I don't know exactly what MathJax cannot handle, so I searched for files containing both ".. math" and "begin{". These were the only instances I found which didn't render correctly with MathJax. I'm afraid it would be almost impossible to search the whole reference manual to look for other problems, since I think they only appear upon rendering in the web browser.

[jhpalmieri]

If this approach (pngs instead of picture environments) looks okay, I'll rebase #6495 on top of it.

[jhpalmieri]

Replying to hivert:

Hi John,

Replying to jhpalmieri:

Great. Please let me know if you have any questions or comments.

I'm running into some problems. Here are what I did: on a fresh 5.4.rc0, I made a clone and imported the two v2 patches. Then I removed the doc/output directory and issued a

sage -docbuild reference html

After that some pages are perfectly Ok but for some other (I can't find a pattern) there is a line in the bottom saying

Loading [MathJax]/fonts/HTML-CSS/TeX/png/imagedata.js

And after somehow ten seconds the text changes to

File failed to load: file:///home/data/Sage-Install/sage-5.4.rc0/devel/sage-doc/doc/output/html/en/reference/_static/fonts/HTML-CSS/TeX/png/imagedata.js

Indeed the file doesn't exists on my filesystem. By the way, I'm using Firefox 15.0 on openSuSE 12.1 64 bits.

I still don't know what's going on with this. On several systems (OS X and sage.math.washington.edu), I don't see this. A few weeks ago, I made a version of the documentation using MathJax and #6495: ​http://sage.math.washington.edu/home/palmieri/misc/6495-jsmath/html/en/index.html. I skimmed through some pages, and don't see any of these problems. Do you have any ideas what could be causing it? I can't imagine why it would make a difference, but did you run sage -b in addition to removing doc/output? Is it repeatable, and are the web pages on which the problems occur repeatable?

[jhpalmieri]

As far as other possibly incompatible pieces of LaTeX in the Sage library, I haven't found any. I ran

sage: search_src('\\\\[a-zA-Z]', interact=False)

to get a list of all strings starting with a backslash and then a letter, then turned this into a list of all of the potential LaTeX commands used in Sage, and compared it to ​MathJax's list. For items on Sage's list that aren't recognized by MathJax, I looked at how they were used in the Sage library. This wasn't a very long list, and I admit that I wasn't extremely careful, but I didn't find any problems.

[jhpalmieri]

(I just fixed a typo in the commit message.)

[dimpase]

Replying to jhpalmieri:

As far as other possibly incompatible pieces of LaTeX in the Sage library, I haven't found any. I ran

sage: search_src('\\\\[a-zA-Z]', interact=False)

to get a list of all strings starting with a backslash and then a letter, then turned this into a list of all of the potential LaTeX commands used in Sage, and compared it to ​MathJax's list. For items on Sage's list that aren't recognized by MathJax, I looked at how they were used in the Sage library. This wasn't a very long list, and I admit that I wasn't extremely careful, but I didn't find any problems.

Here is one thing I found: compare ​http://www.sagemath.org/doc/thematic_tutorials/numtheory_rsa.html and the corresponding page in your output, somewhere in the middle:

Formally, let

Σ={A,B,C,...,Z}

be the set of capital letters of the English alphabet. Furthermore, let

On your pages (and also on my own build with this patch), the formula \Sigma={A,B,C,...,Z} is broken (viewed on OSX10.6.8, with Chrome or with Safari). Namely, it looks like Σ={\textttA,\textttB,\textttC,...,\textttZ}, with \texttt typeset in red.

In the source I see

.. MATH::

    \Sigma
    =
    \{ \texttt{A}, \texttt{B}, \texttt{C}, \dots, \texttt{Z} \}

which look like 100% kosher TeX to me.

Can you fix this in MathJax config?

[dimpase]

Replying to dimpase:

Can you fix this in MathJax config?

This appears to be a mathjax bug, of sorts, according to ​google. If I put the following into Macro: section of mathjax_sage.js in output/html/en/thematic_tutorials/_static/ then it displays just right (textfs and emph are added just for good measure, as they suffer the same fate, apparently).

texttt:             ['\\mathord{\\tt{\\text{#1}}}',1],
textsf:             ['\\mathord{\\sf{\\text{#1}}}',1],
emph:               ['\\textit{#1}',1]

apparently this should be added to the appropriate mathjax_sage.js_t file, although how to rebuild the docs to take care of this change this way, I don't know. If someone tells me this, I could come up with a patch.

[jhpalmieri]

\texttt is supposed to be used in text mode, not math mode, so whether it's a MathJax bug, we should replace it with \mathtt. That fixes the problem for me. I'll add this patch in a few minutes. I also don't know what version of MathJax we have; at some point we should probably upgrade.

[jhpalmieri]

In fact, I think that MathJax is only designed to handle math mode, so we shouldn't expect \texttt or \textsf to work, or \emph in text mode. (Sphinx won't even call MathJax on text not between single backquotes, i.e., on non-math.) Using \emph in math mode is not very common, so I suggest not worrying about it.

[jhpalmieri]

Note that \texttt appears in two more places in the Sage library, but not in the documentation: in combinat/posets/linear_extension.py and in modules/vector_space_morphism.py. In the first of these, it should be replace by \mathtt, and in the second, probably by \text. These changes are not related to this ticket, but we can add them on anyway...

[dimpase]

Replying to jhpalmieri:

Note that \texttt appears in two more places in the Sage library, but not in the documentation: in combinat/posets/linear_extension.py and in modules/vector_space_morphism.py. In the first of these, it should be replace by \mathtt, and in the second, probably by \text. These changes are not related to this ticket, but we can add them on anyway...

Right, let's do this, and hopefully be done with! Indeed, I didn't think straight about \texttt{}.

[jhpalmieri]

optional patch, independent of the others

[jhpalmieri]

If you think we should apply the "optional" patch, please add it to the ticket description (under the list of patches to be applied).

[dimpase]

Replying to jhpalmieri:

If you think we should apply the "optional" patch, please add it to the ticket description (under the list of patches to be applied).

In both cases it's inside the docs for _latex function, which do not appear in the HTML documentation at all, for some reason. I guess sphynx skips stuff beginning from _. I don't know whether it's a bug or a feature.

[jdemeyer]

Sage library

[jdemeyer]

Rebased to sage-5.5.beta1.