# Interface to Mathics

Reported by: Owned by: Sebastian Oehms major sage-9.5 interfaces: optional Mathics Mathematica Wolfram Language interface Sebastian Oehms Dima Pasechnik N/A 5e68c59 5e68c59a618f29aa1d98a515360acbae7ebdb629 #29009 #31676 #31775

Mathics is an open source interpreter for the Wolfram Language. From the introduction of its reference manual:

Mathics — to be pronounced like “Mathematics” without the “emat” — is a general-purpose computer algebra system (CAS). It is meant to be a free, light-weight alternative to Mathematica®. It is free both as in “free beer” and as in “freedom”. There are various online mirrors running Mathics but it is also possible to run Mathics locally. A list of mirrors can be found at the Mathics homepage, http://mathics.github.io.

The programming language of Mathics is meant to resemble Wolfram’s famous Mathematica® as much as possible. However, Mathics is in no way affiliated or supported by Wolfram. Mathics will probably never have the power to compete with Mathematica® in industrial applications; yet, it might be an interesting alternative for educational purposes.

The aim of this ticket is to implement an interface to Mathics. Such an interface has the advantage that functions written in Wolfram Language can be used in Sage on systems that miss a Mathematica® licence.

Moreover, there are many classical software packages written in Wolfram Language that eventually could be used inside Sage in order to participate from their functionality. Currently, for example, Mathics developers are working to get KnotTheory and FeynCalc running.

To install Mathics inside Sage just run:

./configure --enable-download-from-upstream-url
make mathics


Thanks to the developers of Mathics to support this interface!

### comment:1 Changed 17 months ago by Sebastian Oehms

Branch: → u/soehms/mathics_interface

### comment:2 Changed 17 months ago by Sebastian Oehms

Commit: → 85a3a0e5c3a0adf5bc03d9c417db3f5de68ac5d7 → #29009 #31676 #31775

The initial version for the interface is an adaption of the corresponding interface for Mathematica. But in contrast it directly uses the Mathics library functionality instead of Pexpect. It contains all doctests inherited from the Mathematica module but also additional ones which are more Mathics specific. Furthermore, the coverage of doctests is complete.

Two of the Mathematica doctests use FindMinimum which doesn't seem to be available in Mathics. I mark them as not tested. 25 doctests had differences to the output that once has been produced by Mathematica. Most of them are caused by a different way of formatting. I've accepted them all, since they where either minor or a task for Mathics developers to decide if they are acceptable or not (see also this comment in Mathics issue 1235). Explicitly these are:

**********************************************************************
File "src/sage/interfaces/mathics.py", line 37, in sage.interfaces.mathics
Failed example:
mobj = mathics(x^2-1); mobj       # optional - mathics
Expected:
x^2 - 1
Got:
-1 + x ^ 2
**********************************************************************
File "src/sage/interfaces/mathics.py", line 39, in sage.interfaces.mathics
Failed example:
mobj.Factor()                     # optional - mathics
Expected:
(x + 1)*(x - 1)
Got:
(-1 + x) (1 + x)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 51, in sage.interfaces.mathics
Failed example:
mobj2 = mobj.Factor(); mobj2          # optional - mathics
Expected:
(-1 + x)*(1 + x)
Got:
(-1 + x) (1 + x)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 66, in sage.interfaces.mathics
Failed example:
mathics('Factor[x^2-1]')          # optional - mathics
Expected:
(-1 + x)*(1 + x)
Got:
(-1 + x) (1 + x)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 78, in sage.interfaces.mathics
Failed example:
mathics.eval('x^2 - 1')           # optional - mathics
Expected:
2
-1 + x
Got:
'-1 + x ^ 2'
**********************************************************************
File "src/sage/interfaces/mathics.py", line 137, in sage.interfaces.mathics
Failed example:
eqn                               # optional - mathics
Expected:
5 + 3*x == 14
Got:
5 + 3 x == 14
**********************************************************************
File "src/sage/interfaces/mathics.py", line 142, in sage.interfaces.mathics
Failed example:
print(sys)                        # optional - mathics
Expected:
2
{x  - 3 y == 3, 2 x - y == 1}
Got:
{x ^ 2 - 3 y == 3, 2 x - y == 1}
**********************************************************************
File "src/sage/interfaces/mathics.py", line 167, in sage.interfaces.mathics
Failed example:
v = m([eq1, eq2]); v              # optional - mathics
Expected:
{x^2 - 3*y == 3, 2*x - y == 1}
Got:
{x ^ 2 - 3 y == 3, 2 x - y == 1}
**********************************************************************
File "src/sage/interfaces/mathics.py", line 193, in sage.interfaces.mathics
Failed example:
eqn.FindRoot(['x', 2])            # optional - mathics
Expected:
{x -> 1.512134551657842}
Got:
{x -> 1.51213}
**********************************************************************
File "src/sage/interfaces/mathics.py", line 223, in sage.interfaces.mathics
Failed example:
print(g)                           # optional - mathics
Expected:
2       100       101    200
100 + 315 x - 85 x  + 25 x    + 12 x    + x
Got:
100 + 315 x - 85 x ^ 2 + 25 x ^ 100 + 12 x ^ 101 + x ^ 200
**********************************************************************
File "src/sage/interfaces/mathics.py", line 226, in sage.interfaces.mathics
Failed example:
g                                  # optional - mathics
Expected:
100 + 315*x - 85*x^2 + 25*x^100 + 12*x^101 + x^200
Got:
100 + 315 x - 85 x ^ 2 + 25 x ^ 100 + 12 x ^ 101 + x ^ 200
**********************************************************************
File "src/sage/interfaces/mathics.py", line 228, in sage.interfaces.mathics
Failed example:
print(g.Factor())                  # optional - mathics
Expected:
100               100
(20 - 5 x + x   ) (5 + 17 x + x   )
Got:
(5 + 17 x + x ^ 100) (20 - 5 x + x ^ 100)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 235, in sage.interfaces.mathics
Failed example:
print(f.Factor())                  # optional - mathics
Expected:
3                  2    3
x  (x - y) (-2 x + x  + y )
Got:
x ^ 3 (x - y) (-2 x + x ^ 2 + y ^ 3)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 275, in sage.interfaces.mathics
Failed example:
print(x)                  # optional - mathics
Expected:
Pi
--
2
Got:
Pi / 2
**********************************************************************
File "src/sage/interfaces/mathics.py", line 327, in sage.interfaces.mathics
Failed example:
math_bessel_K(2,I)                      # optional - mathics
Expected:
-2.59288617549119697817 + 0.18048997206696202663*I
Got:
-2.5928861754911969782 + 0.18048997206696202663 I
**********************************************************************
File "src/sage/interfaces/mathics.py", line 341, in sage.interfaces.mathics
Failed example:
slist3 = mlist.sage(); slist3         # optional - mathics
Expected:
[[1, 2], 3.00000000000000, I + 4]
Got:
[[1, 2], 3.00000000000000, 4.00000000000000 + 1.00000000000000*I]
**********************************************************************
File "src/sage/interfaces/mathics.py", line 360, in sage.interfaces.mathics
Failed example:
mathics('Pi/2').N(10)           # optional -- mathics
Expected:
1.5707963268
Got:
1.570796327
**********************************************************************
File "src/sage/interfaces/mathics.py", line 362, in sage.interfaces.mathics
Failed example:
mathics('Pi').N(10)             # optional -- mathics
Expected:
3.1415926536
Got:
3.141592654
**********************************************************************
File "src/sage/interfaces/mathics.py", line 364, in sage.interfaces.mathics
Failed example:
mathics('Pi').N(50)             # optional -- mathics
Expected:
3.14159265358979323846264338327950288419716939937511
Got:
3.1415926535897932384626433832795028841971693993751
**********************************************************************
File "src/sage/interfaces/mathics.py", line 366, in sage.interfaces.mathics
Failed example:
str(mathics('Pi*x^2-1/2').N())  # optional -- mathics
Expected:
2
-0.5 + 3.14159 x
Got:
'-0.5 + 3.14159 x ^ 2.'
**********************************************************************
File "src/sage/interfaces/mathics.py", line 670, in sage.interfaces.mathics.Mathics.chdir
Failed example:
mathics('Directory[]')      # optional - mathics
Expected:
"/"
Got:
/
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1015, in sage.interfaces.mathics.MathicsElement._sage_
Failed example:
s = m.sage(); s       # optional - mathics
Expected:
[[1.00000000000000, 4], pi, 3.20000000000000*e100, I]
Got:
[[1.00000000000000, 4], pi, 3.20000000000000*e100, 1.00000000000000*I]
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1019, in sage.interfaces.mathics.MathicsElement._sage_
Failed example:
s[3]^2                # optional - mathics
Expected:
-1
Got:
-1.00000000000000
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1155, in sage.interfaces.mathics.MathicsElement._rich_repr_
Failed example:
P._rich_repr_(dm)                              # optional - mathics
Expected:
OutputImagePng container
Got:
OutputImageSvg container
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1195, in sage.interfaces.mathics.MathicsElement.show
Failed example:
show(Q)                                    # optional - mathics
Expected:
<html><script type="math/tex">\frac{\sin (x \cos (y))}{\sqrt{1-x^2}}</script></html>
Got:
Sin[x Cos[y]] / Sqrt[1 - x ^ 2]
**********************************************************************


Observe that the graphical support of Mathics is restricted to SVG. For the usage in Sage it would be preferable to have PNG possible, as well, or to convert it here to PNG (for example using svg2rlg introducing a new dependency to svglib).

Sage package integration is initialized using:

sage -package create Mathics3 --pypi --type optional


I rename the created package Mathics3 to mathics to match Sage package naming conventions.

New commits:

 ​ea877d9 29009: initial version ​e7d6c32 29009: add comment to doctest ​d06d3d7 Merge branch 'u/soehms/mpmath_divmod_29009' of trac.sagemath.org:sage into mathics_interface ​3e82038 31676: initial version ​507eb62 Merge branch 'u/soehms/mpmath_hash' of trac.sagemath.org:sage into mathics_interface ​e621dea 31775: initial version ​84b8c14 Merge branch 'u/soehms/complex_interfaces_31775' of trac.sagemath.org:sage into mathics_interface ​85a3a0e 31778: initial version

### comment:3 Changed 17 months ago by R. Bernstein

Thanks for undertaking this, Sebastian Oehms!

Know that I'll help out whereever and however I can.

### comment:4 follow-up:  5 Changed 17 months ago by Juan Mauricio Matera

The problem with the order of terms and factors is that usually the evaluation of algebraic expressions is delegated to sympy, so we cannot from Mathics warrant a specific order. A way to make tests compatible with both backends (WMA and Mathics) could be to write tests that check that a difference between two equivalent expressions is zero.

Regarding format, it is possible to use an init file/ set of commands to overwrite the standard behavior of MakeBoxes? over Times or Power.

About images, notice that to produce pngs, you can load one of the optional backend modules: pymathics-matplotlib or pymathics-asymptote. To load the modules, you need to install one of the following libraries:

and then run inside Mathics either LoadModule["pymathics-matplotlib"] or LoadModule["pymathics-asymptote"]. Both are work in progress, but produces pngs for basic plots and graphics.

### comment:5 in reply to:  4 Changed 17 months ago by R. Bernstein

The problem with the order of terms and factors is that usually the evaluation of algebraic expressions is delegated to sympy, so we cannot from Mathics warrant a specific order.

This is related to something I noticed a while ago and have been meaning to fix. I thought I recorded it as an issue and I think I had a local branch for it, but I can't find the issue or branch. The closest I can find is https://github.com/mathics/Mathics/issues/1098 from Jan 3, 2021 and I have just renamed the title to make it more in line with what I have in mind.

We should be canonicalize output along the lines of standard formatting conventions and practice: if I have an expression in a variable, in StandardOutput format the higher powers of the variable come first in order and any constant comes last. It gets a little more complicated when there are combinations of variables.

Interestingly, on parsing we do canonicalize the input seen, and it is in the opposite order! But this is the way WL does it too.

However WL seems to canonicalize its StandardOutput too so you don't notice the reversal between what is parsed in input (as opposed to written by the user) and what is formatted on output.

Were this done, we would have both more natural looking, predicatble, and more WL-compatible output.

Last edited 17 months ago by R. Bernstein (previous) (diff)

### comment:6 follow-up:  8 Changed 17 months ago by Sebastian Oehms

Description: modified (diff)

To be more clear about the differences to the Mathematica results in the doctests: These are not a real problem for this interface. I just listed them for documentation reason before I erased them. So, please don't spend too much time with this.

Concerning the PNG format: I will check this out using your optional backend modules. But, is there something standing against just converting SVG to PNG?

### comment:7 follow-up:  9 Changed 17 months ago by R. Bernstein

With respect to:

But note that the current stable release contains an incompatibility to mpmath running on a Sage backend

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

It is better to have many smaller releases as opposed to larger releases over a longer periods of time, especially as we are in a period of great flux and we are making great strides towards reducing the overall flakiness.

Last edited 17 months ago by R. Bernstein (previous) (diff)

### comment:8 in reply to:  6 ; follow-up:  10 Changed 17 months ago by Juan Mauricio Matera

To be more clear about the differences to the Mathematica results in the doctests: These are not a real problem for this interface. I just listed them for documentation reason before I erased them. So, please don't spend too much time with this.

Concerning the PNG format: I will check this out using your optional backend modules. But, is there something standing against just converting SVG to PNG?

Natively, mathics is able to produce asymptote and SVG graphics, with the first having much better support than the second. Then, to get a png we need to compile the asymptote code to produce the image. But to do that, we need to install a lot of dependencies. The other way around is to use matplotlib, but this requires a new implementation. This is why we have this in external packages.

### comment:9 in reply to:  7 ; follow-up:  11 Changed 17 months ago by Sebastian Oehms

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

That would be great! Thanks!

### comment:10 in reply to:  8 Changed 17 months ago by Sebastian Oehms

Natively, mathics is able to produce asymptote and SVG graphics, with the first having much better support than the second. Then, to get a png we need to compile the asymptote code to produce the image. But to do that, we need to install a lot of dependencies. The other way around is to use matplotlib, but this requires a new implementation. This is why we have this in external packages.

I see! But, I think we can postpone the PNG availability to a follow up ticket. This is not essential to start with the interface. At least, if you convert to Sage objects you can do plotting there.

### comment:11 in reply to:  9 ; follow-up:  12 Changed 17 months ago by R. Bernstein

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

That would be great! Thanks!

Ok tentatively then we can plan on a release the weekend of May 15-16, assuming that Mauricio is on board with this.

### comment:12 in reply to:  11 ; follow-up:  13 Changed 17 months ago by R. Bernstein

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

That would be great! Thanks!

Mathics3 2.2.0 has been released. Hopefully it addresses the problems with the last release. The next release should aslo add FindMinimum.

### comment:13 in reply to:  12 Changed 17 months ago by Sebastian Oehms

Mathics3 2.2.0 has been released. Hopefully it addresses the problems with the last release.

Thanks! These problems are solved. But on sytsems where lxml is not present I'm getting a startup message lxml.html is not available..., now, which distrubes the first doctest:

File "src/sage/interfaces/mathics.py", line 37, in sage.interfaces.mathics
Failed example:
mobj = mathics(x^2-1); mobj       # optional - mathics
Expected:
-1 + x ^ 2
Got:
lxml.html is not available...
-1 + x ^ 2
**********************************************************************
1 of  87 in sage.interfaces.mathics
[222 tests, 1 failure, 9.45 s]


Is this something essential? If not: is there a way to supress this message in MathicsSession? Or, is there a missing dependency?

### comment:14 Changed 17 months ago by R. Bernstein

This dependency was newly added to the last release when an HTML import feature was added.

It shouldn't be necessary. Perhaps down the line, loading various types of files will moved to packages outside of the Mathics core.

When https://github.com/mathics/Mathics/pull/1379 lands, it should address this by making the lxml package optional. And next release we may have a FindMinimum as well.

For now though, I don't know what to suggest. Sigh.

Also, after this gets into sage, we should figure out how to incorporate sage testing into a broader set of tests we do. (We need this for other packages as well.)

### comment:15 follow-up:  16 Changed 17 months ago by R. Bernstein

I just thought of some more possibilities.

1. install lxml in sage. If sage already has something for parsing HTML, then maybe we can use that in the next release of Mathics.
2. If lxml is not installed, provide your own lxml import stub with submodule html and a parse function in it.

### comment:16 in reply to:  15 Changed 17 months ago by Sebastian Oehms

When https://github.com/mathics/Mathics/pull/1379 lands, it should address this by making the lxml package optional. And next release we may have a FindMinimum? as well.

Thats fine!

install lxml in sage ...

If lxml is optional in Mathics then it doesn't make sense to do anything in Sage (especially outside this interface) just to avoid this message. If a Sage user wants to use the optional functionality of Mathics via the interface then he can install it manually by sage -pip install lxml.

If sage already has something for parsing HTML, then maybe we can use that in the next release of Mathics.

AFAIK, html parsers that can be used per default in Sage are those of the following libraries:

sage: import html
sage: import html5lib


### comment:17 Changed 17 months ago by R. Bernstein

Ok. We'll go with one of those, probably html, which comes built in to Python in the version ranges we support, and most likely works just fine.

### comment:18 Changed 17 months ago by git

Branch pushed to git repo; I updated commit sha1. New commits:

 ​5dca2f6 Merge branch 'u/soehms/mathics_interface' of trac.sagemath.org:sage into mathics_interface_31778 ​1051c6f 31778: update Mathics3 to version 2.2.0

### comment:19 Changed 17 months ago by Sebastian Oehms

Authors: → Sebastian Oehms modified (diff)

With the current commit there are three expected doctest failures (one concerning lxml and two with respect to FindMiminum) which sould disappear with the next Mathics release.

### comment:20 Changed 15 months ago by git

Commit: 1051c6f82b6eb6a8aefebef9e6358f45d95f167a → cdd4328915640991c9d829659155da54271d3136

Branch pushed to git repo; I updated commit sha1. New commits:

 ​529a722 Merge branch 'u/soehms/mathics_interface' of trac.sagemath.org:sage into mathics_interface_31778 ​cdd4328 31778: upgrade to Mathics3 3.1.0

### comment:21 Changed 15 months ago by Sebastian Oehms

The upgrade to version 3.1.0 fixes the lxml issue, but not the one with FindMiminum.

### comment:22 Changed 15 months ago by Sebastian Oehms

Description: modified (diff)

### comment:23 Changed 15 months ago by git

Commit: cdd4328915640991c9d829659155da54271d3136 → a30242a2fb89676c454cf16d90340401bc0d52b8

Branch pushed to git repo; I updated commit sha1. New commits:

 ​a30242a 31778: dependency to mathics_scanner added

### comment:24 Changed 14 months ago by Matthias Köppe

Milestone: sage-9.4 → sage-9.5

### comment:25 Changed 13 months ago by git

Commit: a30242a2fb89676c454cf16d90340401bc0d52b8 → 5e68c59a618f29aa1d98a515360acbae7ebdb629

Branch pushed to git repo; I updated commit sha1. New commits:

 ​8cac399 Merge branch 'u/soehms/mathics_interface' of trac.sagemath.org:sage into mathics_interface_31778 ​1565c92 31778: update to Mathics3 4.0.0 ​a95d91c 31778: ignore FindMinimum doctests ​5e68c59 31778: add dependency importlib_metadata (because of Python 3.7)

### comment:26 Changed 13 months ago by Sebastian Oehms

Status: new → needs_review

The commits above contain an update to the current release of Mathics3 introducing two new dependencies to pint and palettable. Still, the Mathematica function FindMinimum is not present in this version. But anyway, I think this is not necessary for a start with the interface. Thus, I marked the corresponding doctests to be ignored.

I've started a workflow which is identical to tox-optional.yml except that the matrix for the optional packages is reduced to mathics.

It seems that there are no failures which are related to the new package except on debian buster one failing doctest on mathics.py. This was caused by the missing dependency to importlib_metadata` which is needed for Sage on Python 3.7 and which is added in the current commit. Rerunning the debian buster test looks good now, as well.

Therefore, I think the ticket is ready for review, right now!

### comment:27 Changed 13 months ago by Dima Pasechnik

Description: modified (diff)

### comment:28 follow-up:  29 Changed 13 months ago by Dima Pasechnik

Reviewers: → Dima Pasechnik needs_review → positive_review

looks good to me, thanks