Opened 10 years ago

# Upgrade the notebook to use MathJax instead of jsMath — at Version 44

Reported by: Owned by: mpatel jason, was major sage-5.4 notebook sd41 rbeezer, robert.marik, jhpalmieri, rkirov, kcrisman, kini Fixed upstream, but not in a stable release.

MathJax is the successor to jsMath.

This will also fix #1608

apply: mathjax-try2.patch This depends on the mathjax branch of my sagenb git fork: https://github.com/jasongrout/sagenb/tree/mathjax

### comment:2 follow-up: ↓ 4 Changed 10 years ago by rbeezer

I've been experimenting a bit with MathJax, outside of the notebook. Mostly thinking about how it will handle tex4ht output in jsMath mode, but here are some some observations that might be useful:

1. Easier in most ways to structure a page to use MathJax. Just insert something like
<script type="text/javascript" src="path-to-MathJax/MathJax.js"></script>


No need for a "process()" call at the end, etc.

1. Default is to not recognize single dollar-signs as delimiters. Alternative is $$..$$. This would be a good thing, since if a user adds text (via TinyMCE) right now jsMath tries to parse the following as math. $$..$$ and $..$ both work for display math. We would break lots of old worksheets if we stopped recognizing $..$.
1. There is a jsMath compatibility mode. I believe I've found one bug in this already (reported upstream). I'd imagine this is not a development priority, but who knows?
1. Modes and configuration is controlled globally by config/MathJax.js which is just one huge well-commented Javascript object. It can be overridden in a web page by adding a new version into the script block mentioned above. Maybe we want to make this easy for users to access, or maybe it is easy already, or maybe we don't want to bother.
1. This looks to me like the best introduction to the types of decisions we will want to make about what to cut over to:
1. MathJax is HUGE. Fonts for lots of Unicode points, I guess. SVN checkout is 53 MB, after unzipping fonts.zip it all occupies 171 MB.
1. I just noticed this morning that html.table() uses class="math" which is a jsMath way to tag span's or div's for processing. I haven't found how to do something similiar in MathJax, though this will work in jsMath compatibility mode. A very small test would indicate that the two modes can be used at the same time.
1. Consonant with (2) and (7), I'd love to see the notebook formatting move to something closer to rigorous XML (ie XHTML, I guess). Certain types of processing would be easier if we did, but that is not really what this ticket is all about.

### comment:3 Changed 10 years ago by robert.marik

Thanks for comment, I feel that MathJax? is much slower on cheap/older computer. This could be an important issue for using Sage in highschools and universities. Is it possible to keep both MathJax? and jsMath and let the notebook admin to choose, which one will be used?

### comment:4 in reply to: ↑ 2 ; follow-up: ↓ 10 Changed 10 years ago by jason

1. MathJax is HUGE. Fonts for lots of Unicode points, I guess. SVN checkout is 53 MB, after unzipping fonts.zip it all occupies 171 MB.

I think MathJax? includes the equivalent of our jsmath-image-fonts spkg. If we added MathJax? to Sage, it might be good to strip out the image fonts and distribute them separately as a mathjax-image-fonts spkg (mathjax faq tells how to do this, I believe). Somewhere I have a half-finished prototype of this solution.

### comment:6 Changed 10 years ago by jason

• Description modified (diff)

### comment:7 Changed 10 years ago by jason

• Description modified (diff)

### comment:8 Changed 10 years ago by jason

• Status changed from new to needs_work

### comment:9 Changed 10 years ago by jason

• Description modified (diff)

### comment:10 in reply to: ↑ 4 ; follow-up: ↓ 11 Changed 10 years ago by rminer

1. MathJax is HUGE. Fonts for lots of Unicode points, I guess. SVN checkout is 53 MB, after unzipping fonts.zip it all occupies 171 MB.

I think MathJax? includes the equivalent of our jsmath-image-fonts spkg. If we added MathJax? to Sage, it might be good to strip out the image fonts and distribute them separately as a mathjax-image-fonts spkg (mathjax faq tells how to do this, I believe). Somewhere I have a half-finished prototype of this solution.

Just in case you haven't thought of this, what Gollum (the GitHub forum) folks did was host the MathJax? fonts on Amazon S3, and then not include them in their distribution. I don't know if that would work for you, but I wanted to point it out.

### comment:11 in reply to: ↑ 10 Changed 10 years ago by jason

1. MathJax is HUGE. Fonts for lots of Unicode points, I guess. SVN checkout is 53 MB, after unzipping fonts.zip it all occupies 171 MB.

I think MathJax? includes the equivalent of our jsmath-image-fonts spkg. If we added MathJax? to Sage, it might be good to strip out the image fonts and distribute them separately as a mathjax-image-fonts spkg (mathjax faq tells how to do this, I believe). Somewhere I have a half-finished prototype of this solution.

Just in case you haven't thought of this, what Gollum (the GitHub forum) folks did was host the MathJax? fonts on Amazon S3, and then not include them in their distribution. I don't know if that would work for you, but I wanted to point it out.

That's a very interesting solution. I suppose we could host them on the main sage webserver, for example. Or maybe Google Code or something like that so we had redundant sources.

### comment:12 Changed 10 years ago by jhpalmieri

I just found the web page http://bitbucket.org/kevindunn/sphinx-extension-mathjax/wiki/Home, which allows the use of Sphinx with MathJax. Assuming it works, we could upgrade all of Sage (not just the notebook but also the docs), to use MathJax instead of jsMath.

### comment:13 Changed 9 years ago by drkirkby

MathJax is very impressive looking. I like the idea the user does not have to have any fonts installed on his computer, which is a major advantage over jsMath

Dave

### Changed 9 years ago by rbeezer

Screenshot of double integral

### Changed 9 years ago by rbeezer

Screenshot of matrix

### comment:14 Changed 9 years ago by rbeezer

Steps that will install MathJax with the new Flask notebook.

1. Install the Flask version of the notebook.
http://code.google.com/r/rkirov-flask/

1. Download v1.1 of MathJax as zip file (from bottom of page).
http://www.mathjax.org/download/

1. Unzip the MathJax? distribution into
$SAGE_ROOT/devel/sagenb/sagenb/data and rename the new directory: mathjax-MathJax-5a7e4d7 (or whatever) to be just the directory: MathJax 1. jsmath compatibility. Edit $SAGE_ROOT/devel/sagenb/sagenb/data/MathJax/config/default.js

by adding "jsMath2jax.js" as the first entry of the "extensions" list, so it becomes
extensions: ["jsMath2jax.js", "tex2jax.js"]

1. Edit
$SAGE_ROOT/devel/sagenb/sagenb/data/sage/html/notebook/base.html  and change <script type="text/javascript" src="/javascript/sage/jsmath.js"></script>  to <script type="text/javascript" src="/static/MathJax/MathJax.js?config=default"></script>  This will install MathJax and the notebook will use it. But the setup is buggy, so needs work. I'm sure there is some configuration on the Sage side and/or the MathJax side that will need changes. • A mix of black and green symbols. • Only renders on a reload - adding new TeX via TinyMCE returns with an error about not finding fonts. • Some stray tags are being rendered, or something. Screenshots attached: $$(2\pi h)^{-d}\iint_{\{H(x,\xi) <\tau\}} dx d\xi$$  Matrix: $\begin{bmatrix}
x^2 & y^2\\
x^3 & \cos(z)
\end{bmatrix}$ ### comment:15 Changed 9 years ago by kcrisman • Cc kcrisman added ### comment:16 Changed 9 years ago by jhpalmieri I'm attaching a draft of a patch for the Sage library. The integration with Sphinx is completely untested, and is probably broken. See the top of the patch file for a list of things to do. ### Changed 9 years ago by rbeezer ### comment:17 Changed 9 years ago by rbeezer Added patch to Flask notebook code to support MathJax, built with Davide Cervone's help. 1. This is a hand-edited patch to recover from an hg mess-up. I think it is OK, but be wary. 2. Fonts are coming from MathJax CDN. Comment in code indicates change for local MathJax installation. 3. <script> tags added by Palmieri's Sage library patch need to be handled with care on notebook side, so notebook does not evaluate them. Right now when "Typeset" box is checked, evaluated cells raise a Javascript error in evaluate_script_tags (or a function with a similar name). ### comment:18 follow-up: ↓ 19 Changed 9 years ago by jason John's patch doesn't apply cleanly to stock 4.7; does it apply to the most recent alpha of 4.7.1? applying mathjax.patch patching file sage/misc/latex.py Hunk #15 succeeded at 1750 with fuzz 2 (offset 2 lines). Hunk #23 FAILED at 2382 1 out of 25 hunks FAILED -- saving rejects to file sage/misc/latex.py.rej patch failed, unable to continue (try -v) patch failed, rejects left in working dir errors during apply, please fix and refresh mathjax.patch  ### comment:19 in reply to: ↑ 18 Changed 9 years ago by rbeezer Replying to jason: John's patch doesn't apply cleanly to stock 4.7; does it apply to the most recent alpha of 4.7.1? Yes. rob@tiger:/sage/dev/devel/sage$ ../../sage -version
| Sage Version 4.7.1.alpha2, Release Date: 2011-06-07                |
* Warning: this is a prerelease version, and it may be unstable.     *
rob@tiger:/sage/dev/devel/sage$hg qimport -P http://trac.sagemath.org/sage_trac/raw-attachment/ticket/9774/mathjax.patch adding mathjax.patch to series file applying mathjax.patch now at: mathjax.patch  ### comment:20 follow-up: ↓ 22 Changed 9 years ago by jason Don't we need the actual MathJax? spkg or files somewhere? Rob: do you have instructions for putting these in the directory, or do we use my half-finished patch mentioned in the description, or something else? ### Changed 9 years ago by rbeezer ### comment:21 Changed 9 years ago by rbeezer Attached a patch to add sagenb/notebook/mathjax.py New file is similar to, but not identical to, sagenb/notebook/jsmath.py We did not delete the latter, but I think it can safely go away. ### comment:22 in reply to: ↑ 20 Changed 9 years ago by rbeezer Replying to jason: Don't we need the actual MathJax? spkg or files somewhere? Rob: do you have instructions for putting these in the directory, or do we use my half-finished patch mentioned in the description, or something else? We need an spkg to be Internet-independent. We did not experiment with your old one. I think it would be best to make a new one. This requires a small edit (once you are sure all works with CDN version). Font suggestions coming up. ### comment:23 Changed 9 years ago by rbeezer Davide Cervone says for an spkg we need only keep font subdirectories called otf, eot, svg  In particular png subdirectories are huge-est and can go away. Rob ### comment:24 Changed 9 years ago by jason ah; I bet it's because the svg fonts replace the png fonts for all browsers we care about, or something. Okay, Davide is the final authority on what browsers support what... ### comment:25 follow-up: ↓ 26 Changed 9 years ago by jason I get an error from the line: response = make_response(render_template('js/mathjax.js', theme_mathjax_macros=mathjax_macros)) in flask_version/base.py. I see a jsmath.js template in sagenb/data/sage/js/jsmath.js template. Is this another file that didn't get added to the patch? ### Changed 9 years ago by rbeezer ### comment:26 in reply to: ↑ 25 Changed 9 years ago by rbeezer Replying to jason: Is this another file that didn't get added to the patch? Yes, same situation - "old" file could be removed. "file-add-two" should create the missing file. Sorry about all the problems - my hg status is useless with the hg mess-up. This file will require an edit when we move from CDN to spkg. Thank-you!!!!!!!!!!!!!! ### comment:27 Changed 9 years ago by jason I have a working version, but there still seems to be something slightly different about Typeset mode compared to before. I'm posting my patch, but there is still *lots* of debugging stuff in the patch that should be taken out before a final version is posted. ### Changed 9 years ago by jason apply to sagenb repository; apply on top of previous patches ### comment:28 Changed 9 years ago by rbeezer The problem with the Sage notebook code intercepting (and evaluating) the contents of <script>,</script> tags can probably be rectified in one, or both, of the functions of the Flask notebook: In sagenb/data/sage/js/notebook_lib.js: eval_script_tags separate_script_tags  ### comment:29 Changed 9 years ago by jason rbeezer: yes, that's what my patch does. The current remaining problem is something different. ### comment:30 Changed 9 years ago by jhpalmieri In the current version of mathjax.patch, look at the top of the file for some unfinished business. For example, the pathname at the end of doc/common/themes/sage/static/mathjax_sage.js_t might be wrong, which might break MathJax? in the Sphinx documentation. ### comment:31 Changed 9 years ago by rbeezer In the current state, the --mathjax switch seems to work with a build of the html version of the reference manual. Mathematics in the documentation is being wrapped with $$,$$ pairs, which are the default MathJax? delimiters for inline mathematics. However these snippets are not being rendered when viewed in a browser. In the page source of a page of the html documentation, I see a script in a file named mathjax_sage.js which we are not generating. We do have jsmath_sage.js lying about in the doc directories, which have the full jsmath configuration information. So this likely needs repair before an HTML documentation page with MathJax? support is built properly. ### comment:32 Changed 9 years ago by rbeezer OK, I can force mathjax_sage.js to be created by totally trashing some of the documentation output with devel/sage/doc/output/html$ rm -rf en


before rebuilding the HTML reference documentation. It comes from a template file mathjax_sage.js_t that one of the patches creates properly.

Mathematics still does not render, but I think the MathJax? configuration is looking locally. Time for a local MathJax? install, I think.

### comment:33 follow-up: ↓ 34 Changed 9 years ago by rbeezer

In a page of the documentation, I inserted manually

<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML-full">


and the mathematics rendered properly with MathJax.

Then I installed the mathjax 1.1a distribution alongside jsmath in the Flask notebook tree and used

<script type="text/javascript" src="http://cdn.mathjax.org/mathjax/latest/MathJax.js?config=TeX-AMS_HTML-full">


which also caused the math to render properly. (/sage/notebook is my SAGE_ROOT)

In both cases the fonts did not look too great to my eye.

Questions:

(a) How do we get these scripts commands to be placed properly in each page of the docs, with a proper relative path?

(b) Is there really a clean separation between Sage and the notebook if --mathjax mode for documentation requires the MathJax code from the notebook?

### comment:34 in reply to: ↑ 33 Changed 9 years ago by rbeezer

Then I installed the mathjax 1.1a distribution alongside jsmath in the Flask notebook tree and

which also caused the math to render properly. (/sage/notebook is my SAGE_ROOT)

Oops, the incantation for a local install of MathJax should have been

<script type="text/javascript" src="/sage/notebook/devel/rkirov-flask/sagenb/data/mathjax/MathJax.js?config=TeX-AMS_HTML-full"></script>


### comment:35 Changed 9 years ago by jhpalmieri

Here's a new version of my patch. With a locally installed version of MathJax, the documentation seems to render properly. I don't know if this is the best solution, but at least it seems to work.

(I installed MathJax by downloading a zip file: https://github.com/mathjax/MathJax/zipball/v1.1a. I unzipped it and renamed the resulting directory to mathjax, and moved it to SAGE_ROOT/local/lib/python/site-packages/sagenb-0.9.0-py2.6.egg/sagenb/data/, alongside of the old jsmath directory. I also put a copy of the mathjax directory in SAGE_ROOT/devel/sagenb/sagenb/data/. I haven't tried to install local fonts.)

### Changed 9 years ago by jhpalmieri

(experimental) patch for Sage library

### comment:36 Changed 9 years ago by jhpalmieri

By the way, one comment about the latest version of my patch: it fixes the problem Rob mentioned about getting the appropriate invocation to MathJax in each piece of documentation. It does this by setting the variable mathjax_path to

file://[path to MathJax.js]?config=TeX-AMS_HTML-full,file://[path to mathjax_sage.js]


The paths here are absolute ones, in SAGE_ROOT/devel/sage/doc/output/html/en/_static/. This directory is created by sage -docbuild website html, so now if you build the docs with the -j flag (to enable MathJax), it first builds the website target, then builds whatever you requested. There might be a better way to do this, using relative paths, but I couldn't get it to work. For example, using

    <script type="text/javascript" src="_static/MathJax.js?config=TeX-AMS_HTML-full"></script>


successfully turns on MathJax, but without the local configuration turning on the custom macros. Using

    <script type="text/javascript" src="_static/MathJax.js?config=TeX-AMS_HTML-full,mathjax_sage.js"></script>


doesn't work, nor does any variation of it that I tried. (Note that when you change the path specifying the local configuration file, you also need to change the last line of that configuration file to match it, according to http://www.mathjax.org/docs/1.1/configuration.html.) If we could copy the file mathjax_sage.js to _static/config/local/, then we could use

    <script type="text/javascript" src="_static/MathJax.js?config=TeX-AMS_HTML-full,local/mathjax_sage.js"></script>


for the invocation, and

MathJax.Ajax.loadComplete("[MathJax]/config/local/mathjax_sage.js")


for the last line in mathjax_sage.js. But I don't know how to put this file in the right place: it's autogenerated, so we can't put it in the MathJax local directory ahead of time, and I can't figure out how to get Sphinx to do it when it's building the documentation.

Another option would be to try to get Sphinx to write this into each html file:

<script type="text/x-mathjax-config">
[contents of mathjax_sage.js]
</script>
<script type="text/javascript" src="_static/MathJax.js?config=TeX-AMS_HTML-full"></script>


We can get the last line by specifing mathjax_path in conf.py, but I don't know how to get the first part.

I'll keep working on it...

### comment:37 Changed 9 years ago by rbeezer

Hi John,

Are you testing with the Flask notebook? If not, it might be easier to build what look like absolute paths to the MathJax files in the data directory of the notebook, because of the way Flask sets up paths with the "route()" command? I can't tell if this will solve the problem you are wrestling with or not.

The <script> tags were being messed with by the notebook on the server side just before being dispatched. But then the client sees two versions - wrapped and unwrapped, and totally trashes one of them to prevent running a script twice. I thought Jason had a patch for this too, but I haven't seen it. The symptom would be when you cycle through wrap, no-wrap, and hide for some output, then one of them is missing. I guess we can press on and fix this eventually.

About to go off-line shortly for the long weekend, but will come back to this when I can stick with it.

Rob

### comment:38 Changed 9 years ago by jhpalmieri

For building the documentation, the notebook isn't relevant: they get built from the command line. I think that when trying to make MathJax work with the notebook, we'll want to construct the URLs differently than I'm doing for the docs, but I don't know enough about the paths, etc., in the notebook to say exactly how to do it. For what it's worth, though, I have been using the flask notebook.

### comment:39 Changed 9 years ago by jason

I've uploaded my current flask patch queue here: http://sage.math.washington.edu/home/jason/mathjax-flask-patches.tar.gz My current (probably non-working) patch queue for my sage library, including two mathjax patches, is here: http://sage.math.washington.edu/home/jason/mathjax-sage-patches.tar.gz

These are both works-in-progress, but currently stalled for at least a week or two.

### comment:40 Changed 9 years ago by jason

I stalled working on it because of other time pressures with the single-cell, but also because I realized that the *next* version of Sphinx supports Mathjax out of the box, while I thought the current version would require patches.

### comment:41 Changed 9 years ago by jhpalmieri

I think that the implementation of MathJax in Sphinx comes from the file mathjax.py, which is available for download separately. So my patch just adds it.

I'm attaching another version ("v2") of my patch; this one writes the MathJax local configuration file to a file in SAGE_ROOT/local/lib/python/site-libraries/sagenb.../, which may not be a good idea. But it allows for a simpler invocation of MathJax. I'm really not sure which approach is better.

### Changed 9 years ago by jhpalmieri

(experimental) patch for Sage library, alternate version

### comment:42 Changed 9 years ago by jason

What is the current status on this now? Where should we start working on it again?

### comment:43 Changed 9 years ago by rkirov

The last patch involving MathJax?, that I rebased from the ones here is on google code:

Unless Jonathan Gutow did something else on top of it, this is latest work.

### Changed 9 years ago by jason

Rebased version of comprehensive mathjax patch to 4.8.alpha4

### Changed 9 years ago by jason

Apply only this patch

### comment:44 Changed 9 years ago by jason

• Description modified (diff)
• Milestone set to sage-5.0
Note: See TracTickets for help on using tickets.