#31778 closed enhancement (fixed)

Interface to Mathics

Reported by: Sebastian Oehms Owned by:
Priority: major Milestone: sage-9.5
Component: interfaces: optional Keywords: Mathics Mathematica Wolfram Language interface
Cc: Merged in:
Authors: Sebastian Oehms Reviewers: Dima Pasechnik
Report Upstream: N/A Work issues:
Branch: 5e68c59 (Commits, GitHub, GitLab) Commit: 5e68c59a618f29aa1d98a515360acbae7ebdb629
Dependencies: #29009 #31676 #31775 Stopgaps:

Status badges

Description (last modified by Dima Pasechnik)

Mathics is an open source interpreter for the Wolfram Language. From the introduction of its reference manual:

Mathics — to be pronounced like “Mathematics” without the “emat” — is a general-purpose computer algebra system (CAS). It is meant to be a free, light-weight alternative to Mathematica®. It is free both as in “free beer” and as in “freedom”. There are various online mirrors running Mathics but it is also possible to run Mathics locally. A list of mirrors can be found at the Mathics homepage, http://mathics.github.io.

The programming language of Mathics is meant to resemble Wolfram’s famous Mathematica® as much as possible. However, Mathics is in no way affiliated or supported by Wolfram. Mathics will probably never have the power to compete with Mathematica® in industrial applications; yet, it might be an interesting alternative for educational purposes.

The aim of this ticket is to implement an interface to Mathics. Such an interface has the advantage that functions written in Wolfram Language can be used in Sage on systems that miss a Mathematica® licence.

Moreover, there are many classical software packages written in Wolfram Language that eventually could be used inside Sage in order to participate from their functionality. Currently, for example, Mathics developers are working to get KnotTheory and FeynCalc running.

To install Mathics inside Sage just run:

./configure --enable-download-from-upstream-url
make mathics

Thanks to the developers of Mathics to support this interface!

Change History (30)

comment:1 Changed 17 months ago by Sebastian Oehms

Branch: u/soehms/mathics_interface

comment:2 Changed 17 months ago by Sebastian Oehms

Commit: 85a3a0e5c3a0adf5bc03d9c417db3f5de68ac5d7
Dependencies: #29009 #31676 #31775

The initial version for the interface is an adaption of the corresponding interface for Mathematica. But in contrast it directly uses the Mathics library functionality instead of Pexpect. It contains all doctests inherited from the Mathematica module but also additional ones which are more Mathics specific. Furthermore, the coverage of doctests is complete.

Two of the Mathematica doctests use FindMinimum which doesn't seem to be available in Mathics. I mark them as not tested. 25 doctests had differences to the output that once has been produced by Mathematica. Most of them are caused by a different way of formatting. I've accepted them all, since they where either minor or a task for Mathics developers to decide if they are acceptable or not (see also this comment in Mathics issue 1235). Explicitly these are:

**********************************************************************
File "src/sage/interfaces/mathics.py", line 37, in sage.interfaces.mathics
Failed example:
    mobj = mathics(x^2-1); mobj       # optional - mathics
Expected:
    x^2 - 1
Got:
    -1 + x ^ 2
**********************************************************************
File "src/sage/interfaces/mathics.py", line 39, in sage.interfaces.mathics
Failed example:
    mobj.Factor()                     # optional - mathics
Expected:
    (x + 1)*(x - 1)
Got:
    (-1 + x) (1 + x)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 51, in sage.interfaces.mathics
Failed example:
    mobj2 = mobj.Factor(); mobj2          # optional - mathics
Expected:
    (-1 + x)*(1 + x)
Got:
    (-1 + x) (1 + x)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 66, in sage.interfaces.mathics
Failed example:
    mathics('Factor[x^2-1]')          # optional - mathics
Expected:
    (-1 + x)*(1 + x)
Got:
    (-1 + x) (1 + x)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 78, in sage.interfaces.mathics
Failed example:
    mathics.eval('x^2 - 1')           # optional - mathics
Expected:
                   2
             -1 + x
Got:
    '-1 + x ^ 2'
**********************************************************************
File "src/sage/interfaces/mathics.py", line 137, in sage.interfaces.mathics
Failed example:
    eqn                               # optional - mathics
Expected:
    5 + 3*x == 14
Got:
    5 + 3 x == 14
**********************************************************************
File "src/sage/interfaces/mathics.py", line 142, in sage.interfaces.mathics
Failed example:
    print(sys)                        # optional - mathics
Expected:
               2
             {x  - 3 y == 3, 2 x - y == 1}
Got:
    {x ^ 2 - 3 y == 3, 2 x - y == 1}
**********************************************************************
File "src/sage/interfaces/mathics.py", line 167, in sage.interfaces.mathics
Failed example:
    v = m([eq1, eq2]); v              # optional - mathics
Expected:
    {x^2 - 3*y == 3, 2*x - y == 1}
Got:
    {x ^ 2 - 3 y == 3, 2 x - y == 1}
**********************************************************************
File "src/sage/interfaces/mathics.py", line 193, in sage.interfaces.mathics
Failed example:
    eqn.FindRoot(['x', 2])            # optional - mathics
Expected:
    {x -> 1.512134551657842}
Got:
    {x -> 1.51213}
**********************************************************************
File "src/sage/interfaces/mathics.py", line 223, in sage.interfaces.mathics
Failed example:
    print(g)                           # optional - mathics
Expected:
                               2       100       101    200
             100 + 315 x - 85 x  + 25 x    + 12 x    + x
Got:
    100 + 315 x - 85 x ^ 2 + 25 x ^ 100 + 12 x ^ 101 + x ^ 200
**********************************************************************
File "src/sage/interfaces/mathics.py", line 226, in sage.interfaces.mathics
Failed example:
    g                                  # optional - mathics
Expected:
    100 + 315*x - 85*x^2 + 25*x^100 + 12*x^101 + x^200
Got:
    100 + 315 x - 85 x ^ 2 + 25 x ^ 100 + 12 x ^ 101 + x ^ 200
**********************************************************************
File "src/sage/interfaces/mathics.py", line 228, in sage.interfaces.mathics
Failed example:
    print(g.Factor())                  # optional - mathics
Expected:
                          100               100
             (20 - 5 x + x   ) (5 + 17 x + x   )
Got:
    (5 + 17 x + x ^ 100) (20 - 5 x + x ^ 100)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 235, in sage.interfaces.mathics
Failed example:
    print(f.Factor())                  # optional - mathics
Expected:
              3                  2    3
             x  (x - y) (-2 x + x  + y )
Got:
    x ^ 3 (x - y) (-2 x + x ^ 2 + y ^ 3)
**********************************************************************
File "src/sage/interfaces/mathics.py", line 275, in sage.interfaces.mathics
Failed example:
    print(x)                  # optional - mathics
Expected:
             Pi
             --
             2
Got:
    Pi / 2
**********************************************************************
File "src/sage/interfaces/mathics.py", line 327, in sage.interfaces.mathics
Failed example:
    math_bessel_K(2,I)                      # optional - mathics
Expected:
    -2.59288617549119697817 + 0.18048997206696202663*I
Got:
    -2.5928861754911969782 + 0.18048997206696202663 I
**********************************************************************
File "src/sage/interfaces/mathics.py", line 341, in sage.interfaces.mathics
Failed example:
    slist3 = mlist.sage(); slist3         # optional - mathics
Expected:
    [[1, 2], 3.00000000000000, I + 4]
Got:
    [[1, 2], 3.00000000000000, 4.00000000000000 + 1.00000000000000*I]
**********************************************************************
File "src/sage/interfaces/mathics.py", line 360, in sage.interfaces.mathics
Failed example:
    mathics('Pi/2').N(10)           # optional -- mathics
Expected:
    1.5707963268
Got:
    1.570796327
**********************************************************************
File "src/sage/interfaces/mathics.py", line 362, in sage.interfaces.mathics
Failed example:
    mathics('Pi').N(10)             # optional -- mathics
Expected:
    3.1415926536
Got:
    3.141592654
**********************************************************************
File "src/sage/interfaces/mathics.py", line 364, in sage.interfaces.mathics
Failed example:
    mathics('Pi').N(50)             # optional -- mathics
Expected:
    3.14159265358979323846264338327950288419716939937511
Got:
    3.1415926535897932384626433832795028841971693993751
**********************************************************************
File "src/sage/interfaces/mathics.py", line 366, in sage.interfaces.mathics
Failed example:
    str(mathics('Pi*x^2-1/2').N())  # optional -- mathics
Expected:
                    2
    -0.5 + 3.14159 x
Got:
    '-0.5 + 3.14159 x ^ 2.'
**********************************************************************
File "src/sage/interfaces/mathics.py", line 670, in sage.interfaces.mathics.Mathics.chdir
Failed example:
    mathics('Directory[]')      # optional - mathics
Expected:
    "/"
Got:
    /
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1015, in sage.interfaces.mathics.MathicsElement._sage_
Failed example:
    s = m.sage(); s       # optional - mathics
Expected:
    [[1.00000000000000, 4], pi, 3.20000000000000*e100, I]
Got:
    [[1.00000000000000, 4], pi, 3.20000000000000*e100, 1.00000000000000*I]
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1019, in sage.interfaces.mathics.MathicsElement._sage_
Failed example:
    s[3]^2                # optional - mathics
Expected:
    -1
Got:
    -1.00000000000000
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1155, in sage.interfaces.mathics.MathicsElement._rich_repr_
Failed example:
    P._rich_repr_(dm)                              # optional - mathics
Expected:
    OutputImagePng container
Got:
    OutputImageSvg container
**********************************************************************
File "src/sage/interfaces/mathics.py", line 1195, in sage.interfaces.mathics.MathicsElement.show
Failed example:
    show(Q)                                    # optional - mathics
Expected:
    <html><script type="math/tex">\frac{\sin (x \cos (y))}{\sqrt{1-x^2}}</script></html>
Got:
    Sin[x Cos[y]] / Sqrt[1 - x ^ 2]
**********************************************************************

Observe that the graphical support of Mathics is restricted to SVG. For the usage in Sage it would be preferable to have PNG possible, as well, or to convert it here to PNG (for example using svg2rlg introducing a new dependency to svglib).

Sage package integration is initialized using:

sage -package create Mathics3 --pypi --type optional

I rename the created package Mathics3 to mathics to match Sage package naming conventions.


New commits:

ea877d929009: initial version
e7d6c3229009: add comment to doctest
d06d3d7Merge branch 'u/soehms/mpmath_divmod_29009' of trac.sagemath.org:sage into mathics_interface
3e8203831676: initial version
507eb62Merge branch 'u/soehms/mpmath_hash' of trac.sagemath.org:sage into mathics_interface
e621dea31775: initial version
84b8c14Merge branch 'u/soehms/complex_interfaces_31775' of trac.sagemath.org:sage into mathics_interface
85a3a0e31778: initial version

comment:3 Changed 17 months ago by R. Bernstein

Thanks for undertaking this, Sebastian Oehms!

Know that I'll help out whereever and however I can.

comment:4 Changed 17 months ago by Juan Mauricio Matera

The problem with the order of terms and factors is that usually the evaluation of algebraic expressions is delegated to sympy, so we cannot from Mathics warrant a specific order. A way to make tests compatible with both backends (WMA and Mathics) could be to write tests that check that a difference between two equivalent expressions is zero.

Regarding format, it is possible to use an init file/ set of commands to overwrite the standard behavior of MakeBoxes? over Times or Power.

About images, notice that to produce png`s, you can load one of the optional backend modules: pymathics-matplotlib or pymathics-asymptote. To load the modules, you need to install one of the following libraries:

https://github.com/mathics3/pymathics-matplotlib https://github.com/mathics3/pymathics-asymptote

and then run inside Mathics either LoadModule["pymathics-matplotlib"] or LoadModule["pymathics-asymptote"]. Both are work in progress, but produces pngs for basic plots and graphics.

comment:5 in reply to:  4 Changed 17 months ago by R. Bernstein

Replying to gh-mmatera:

The problem with the order of terms and factors is that usually the evaluation of algebraic expressions is delegated to sympy, so we cannot from Mathics warrant a specific order.

This is related to something I noticed a while ago and have been meaning to fix. I thought I recorded it as an issue and I think I had a local branch for it, but I can't find the issue or branch. The closest I can find is https://github.com/mathics/Mathics/issues/1098 from Jan 3, 2021 and I have just renamed the title to make it more in line with what I have in mind.

We should be canonicalize output along the lines of standard formatting conventions and practice: if I have an expression in a variable, in StandardOutput format the higher powers of the variable come first in order and any constant comes last. It gets a little more complicated when there are combinations of variables.

Interestingly, on parsing we do canonicalize the input seen, and it is in the opposite order! But this is the way WL does it too.

However WL seems to canonicalize its StandardOutput too so you don't notice the reversal between what is parsed in input (as opposed to written by the user) and what is formatted on output.

Were this done, we would have both more natural looking, predicatble, and more WL-compatible output.

Last edited 17 months ago by R. Bernstein (previous) (diff)

comment:6 Changed 17 months ago by Sebastian Oehms

Description: modified (diff)

To be more clear about the differences to the Mathematica results in the doctests: These are not a real problem for this interface. I just listed them for documentation reason before I erased them. So, please don't spend too much time with this.

Concerning the PNG format: I will check this out using your optional backend modules. But, is there something standing against just converting SVG to PNG?

comment:7 Changed 17 months ago by R. Bernstein

With respect to:

But note that the current stable release contains an incompatibility to mpmath running on a Sage backend

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

It is better to have many smaller releases as opposed to larger releases over a longer periods of time, especially as we are in a period of great flux and we are making great strides towards reducing the overall flakiness.

Last edited 17 months ago by R. Bernstein (previous) (diff)

comment:8 in reply to:  6 ; Changed 17 months ago by Juan Mauricio Matera

Replying to soehms:

To be more clear about the differences to the Mathematica results in the doctests: These are not a real problem for this interface. I just listed them for documentation reason before I erased them. So, please don't spend too much time with this.

Concerning the PNG format: I will check this out using your optional backend modules. But, is there something standing against just converting SVG to PNG?

Natively, mathics is able to produce asymptote and SVG graphics, with the first having much better support than the second. Then, to get a png we need to compile the asymptote code to produce the image. But to do that, we need to install a lot of dependencies. The other way around is to use matplotlib, but this requires a new implementation. This is why we have this in external packages.

comment:9 in reply to:  7 ; Changed 17 months ago by Sebastian Oehms

Replying to gh-rocky:

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

That would be great! Thanks!

comment:10 in reply to:  8 Changed 17 months ago by Sebastian Oehms

Replying to gh-mmatera:

Natively, mathics is able to produce asymptote and SVG graphics, with the first having much better support than the second. Then, to get a png we need to compile the asymptote code to produce the image. But to do that, we need to install a lot of dependencies. The other way around is to use matplotlib, but this requires a new implementation. This is why we have this in external packages.

I see! But, I think we can postpone the PNG availability to a follow up ticket. This is not essential to start with the interface. At least, if you convert to Sage objects you can do plotting there.

comment:11 in reply to:  9 ; Changed 17 months ago by R. Bernstein

Replying to soehms:

Replying to gh-rocky:

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

That would be great! Thanks!

Ok tentatively then we can plan on a release the weekend of May 15-16, assuming that Mauricio is on board with this.

comment:12 in reply to:  11 ; Changed 17 months ago by R. Bernstein

Replying to gh-rocky:

Replying to soehms:

Replying to gh-rocky:

If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.

That would be great! Thanks!

Mathics3 2.2.0 has been released. Hopefully it addresses the problems with the last release. The next release should aslo add FindMinimum.

comment:13 in reply to:  12 Changed 17 months ago by Sebastian Oehms

Replying to gh-rocky:

Mathics3 2.2.0 has been released. Hopefully it addresses the problems with the last release.

Thanks! These problems are solved. But on sytsems where lxml is not present I'm getting a startup message lxml.html is not available..., now, which distrubes the first doctest:

File "src/sage/interfaces/mathics.py", line 37, in sage.interfaces.mathics
Failed example:
    mobj = mathics(x^2-1); mobj       # optional - mathics
Expected:
    -1 + x ^ 2
Got:
    lxml.html is not available...
    -1 + x ^ 2
**********************************************************************
1 item had failures:
   1 of  87 in sage.interfaces.mathics
    [222 tests, 1 failure, 9.45 s]

Is this something essential? If not: is there a way to supress this message in MathicsSession? Or, is there a missing dependency?

comment:14 Changed 17 months ago by R. Bernstein

This dependency was newly added to the last release when an HTML import feature was added.

It shouldn't be necessary. Perhaps down the line, loading various types of files will moved to packages outside of the Mathics core.

When https://github.com/mathics/Mathics/pull/1379 lands, it should address this by making the lxml package optional. And next release we may have a FindMinimum as well.

For now though, I don't know what to suggest. Sigh.

Also, after this gets into sage, we should figure out how to incorporate sage testing into a broader set of tests we do. (We need this for other packages as well.)

comment:15 Changed 17 months ago by R. Bernstein

I just thought of some more possibilities.

  1. install lxml in sage. If sage already has something for parsing HTML, then maybe we can use that in the next release of Mathics.
  2. If lxml is not installed, provide your own lxml import stub with submodule html and a parse function in it.

comment:16 in reply to:  15 Changed 17 months ago by Sebastian Oehms

Replying to gh-rocky:

When https://github.com/mathics/Mathics/pull/1379 lands, it should address this by making the lxml package optional. And next release we may have a FindMinimum? as well.

Thats fine!

install lxml in sage ...

If lxml is optional in Mathics then it doesn't make sense to do anything in Sage (especially outside this interface) just to avoid this message. If a Sage user wants to use the optional functionality of Mathics via the interface then he can install it manually by sage -pip install lxml.

If sage already has something for parsing HTML, then maybe we can use that in the next release of Mathics.

AFAIK, html parsers that can be used per default in Sage are those of the following libraries:

sage: import html
sage: import html5lib

comment:17 Changed 17 months ago by R. Bernstein

Ok. We'll go with one of those, probably html, which comes built in to Python in the version ranges we support, and most likely works just fine.

comment:18 Changed 17 months ago by git

Commit: 85a3a0e5c3a0adf5bc03d9c417db3f5de68ac5d71051c6f82b6eb6a8aefebef9e6358f45d95f167a

Branch pushed to git repo; I updated commit sha1. New commits:

5dca2f6Merge branch 'u/soehms/mathics_interface' of trac.sagemath.org:sage into mathics_interface_31778
1051c6f31778: update Mathics3 to version 2.2.0

comment:19 Changed 17 months ago by Sebastian Oehms

Authors: Sebastian Oehms
Description: modified (diff)

With the current commit there are three expected doctest failures (one concerning lxml and two with respect to FindMiminum) which sould disappear with the next Mathics release.

comment:20 Changed 15 months ago by git

Commit: 1051c6f82b6eb6a8aefebef9e6358f45d95f167acdd4328915640991c9d829659155da54271d3136

Branch pushed to git repo; I updated commit sha1. New commits:

529a722Merge branch 'u/soehms/mathics_interface' of trac.sagemath.org:sage into mathics_interface_31778
cdd432831778: upgrade to Mathics3 3.1.0

comment:21 Changed 15 months ago by Sebastian Oehms

The upgrade to version 3.1.0 fixes the lxml issue, but not the one with FindMiminum.

comment:22 Changed 15 months ago by Sebastian Oehms

Description: modified (diff)

comment:23 Changed 15 months ago by git

Commit: cdd4328915640991c9d829659155da54271d3136a30242a2fb89676c454cf16d90340401bc0d52b8

Branch pushed to git repo; I updated commit sha1. New commits:

a30242a31778: dependency to mathics_scanner added

comment:24 Changed 14 months ago by Matthias Köppe

Milestone: sage-9.4sage-9.5

comment:25 Changed 13 months ago by git

Commit: a30242a2fb89676c454cf16d90340401bc0d52b85e68c59a618f29aa1d98a515360acbae7ebdb629

Branch pushed to git repo; I updated commit sha1. New commits:

8cac399Merge branch 'u/soehms/mathics_interface' of trac.sagemath.org:sage into mathics_interface_31778
1565c9231778: update to Mathics3 4.0.0
a95d91c31778: ignore FindMinimum doctests
5e68c5931778: add dependency importlib_metadata (because of Python 3.7)

comment:26 Changed 13 months ago by Sebastian Oehms

Status: newneeds_review

The commits above contain an update to the current release of Mathics3 introducing two new dependencies to pint and palettable. Still, the Mathematica function FindMinimum is not present in this version. But anyway, I think this is not necessary for a start with the interface. Thus, I marked the corresponding doctests to be ignored.

I've started a workflow which is identical to tox-optional.yml except that the matrix for the optional packages is reduced to mathics.

It seems that there are no failures which are related to the new package except on debian buster one failing doctest on mathics.py. This was caused by the missing dependency to importlib_metadata which is needed for Sage on Python 3.7 and which is added in the current commit. Rerunning the debian buster test looks good now, as well.

Therefore, I think the ticket is ready for review, right now!

comment:27 Changed 13 months ago by Dima Pasechnik

Description: modified (diff)

comment:28 Changed 13 months ago by Dima Pasechnik

Reviewers: Dima Pasechnik
Status: needs_reviewpositive_review

looks good to me, thanks

comment:29 in reply to:  28 Changed 13 months ago by Sebastian Oehms

Replying to dimpase:

looks good to me, thanks

Many thanks, as well!

comment:30 Changed 13 months ago by Volker Braun

Branch: u/soehms/mathics_interface5e68c59a618f29aa1d98a515360acbae7ebdb629
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.