Opened 21 months ago
Closed 17 months ago
#31778 closed enhancement (fixed)
Interface to Mathics
Reported by:  Sebastian Oehms  Owned by:  

Priority:  major  Milestone:  sage9.5 
Component:  interfaces: optional  Keywords:  Mathics Mathematica Wolfram Language interface 
Cc:  Merged in:  
Authors:  Sebastian Oehms  Reviewers:  Dima Pasechnik 
Report Upstream:  N/A  Work issues:  
Branch:  5e68c59 (Commits, GitHub, GitLab)  Commit:  5e68c59a618f29aa1d98a515360acbae7ebdb629 
Dependencies:  #29009 #31676 #31775  Stopgaps: 
Description (last modified by )
Mathics is an open source interpreter for the Wolfram Language. From the introduction of its reference manual:
Mathics — to be pronounced like “Mathematics” without the “emat” — is a generalpurpose computer algebra system (CAS). It is meant to be a free, lightweight alternative to Mathematica®. It is free both as in “free beer” and as in “freedom”. There are various online mirrors running Mathics but it is also possible to run Mathics locally. A list of mirrors can be found at the Mathics homepage, http://mathics.github.io.
The programming language of Mathics is meant to resemble Wolfram’s famous Mathematica® as much as possible. However, Mathics is in no way affiliated or supported by Wolfram. Mathics will probably never have the power to compete with Mathematica® in industrial applications; yet, it might be an interesting alternative for educational purposes.
The aim of this ticket is to implement an interface to Mathics. Such an interface has the advantage that functions written in Wolfram Language can be used in Sage on systems that miss a Mathematica® licence.
Moreover, there are many classical software packages written in Wolfram Language that eventually could be used inside Sage in order to participate from their functionality. Currently, for example, Mathics developers are working to get KnotTheory and FeynCalc running.
To install Mathics inside Sage just run:
./configure enabledownloadfromupstreamurl make mathics
Thanks to the developers of Mathics to support this interface!
Change History (30)
comment:1 Changed 21 months ago by
Branch:  → u/soehms/mathics_interface 

comment:2 Changed 21 months ago by
Commit:  → 85a3a0e5c3a0adf5bc03d9c417db3f5de68ac5d7 

Dependencies:  → #29009 #31676 #31775 
comment:3 Changed 21 months ago by
Thanks for undertaking this, Sebastian Oehms!
Know that I'll help out whereever and however I can.
comment:4 followup: 5 Changed 21 months ago by
The problem with the order of terms and factors is that usually the evaluation of algebraic expressions is delegated to sympy, so we cannot from Mathics warrant a specific order. A way to make tests compatible with both backends (WMA and Mathics) could be to write tests that check that a difference between two equivalent expressions is zero.
Regarding format, it is possible to use an init file/ set of commands to overwrite the standard behavior of MakeBoxes? over Times
or Power
.
About images, notice that to produce png`s, you can load one of the optional backend modules: pymathicsmatplotlib or pymathicsasymptote. To load the modules, you need to install one of the following libraries:
https://github.com/mathics3/pymathicsmatplotlib https://github.com/mathics3/pymathicsasymptote
and then run inside Mathics either LoadModule["pymathicsmatplotlib"]
or
LoadModule["pymathicsasymptote"]
.
Both are work in progress, but produces pngs for basic plots and graphics.
comment:5 Changed 21 months ago by
Replying to ghmmatera:
The problem with the order of terms and factors is that usually the evaluation of algebraic expressions is delegated to sympy, so we cannot from Mathics warrant a specific order.
This is related to something I noticed a while ago and have been meaning to fix. I thought I recorded it as an issue and I think I had a local branch for it, but I can't find the issue or branch. The closest I can find is https://github.com/mathics/Mathics/issues/1098 from Jan 3, 2021 and I have just renamed the title to make it more in line with what I have in mind.
We should be canonicalize output along the lines of standard formatting conventions and practice: if I have an expression in a variable, in StandardOutput
format the higher powers of the variable come first in order and any constant comes last. It gets a little more complicated when there are combinations of variables.
Interestingly, on parsing we do canonicalize the input seen, and it is in the opposite order! But this is the way WL does it too.
However WL seems to canonicalize its StandardOutput
too so you don't notice the reversal between what is parsed in input (as opposed to written by the user) and what is formatted on output.
Were this done, we would have both more natural looking, predicatble, and more WLcompatible output.
comment:6 followup: 8 Changed 21 months ago by
Description:  modified (diff) 

To be more clear about the differences to the Mathematica results in the doctests: These are not a real problem for this interface. I just listed them for documentation reason before I erased them. So, please don't spend too much time with this.
Concerning the PNG
format: I will check this out using your optional backend modules. But, is there something standing against just converting SVG
to PNG
?
comment:7 followup: 9 Changed 21 months ago by
With respect to:
But note that the current stable release contains an incompatibility to mpmath running on a Sage backend
If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.
It is better to have many smaller releases as opposed to larger releases over a longer periods of time, especially as we are in a period of great flux and we are making great strides towards reducing the overall flakiness.
comment:8 followup: 10 Changed 21 months ago by
Replying to soehms:
To be more clear about the differences to the Mathematica results in the doctests: These are not a real problem for this interface. I just listed them for documentation reason before I erased them. So, please don't spend too much time with this.
Concerning the
PNG
format: I will check this out using your optional backend modules. But, is there something standing against just convertingSVG
toPNG
?
Natively, mathics is able to produce asymptote and SVG graphics, with the first having much better support than the second. Then, to get a png we need to compile the asymptote code to produce the image. But to do that, we need to install a lot of dependencies. The other way around is to use matplotlib, but this requires a new implementation. This is why we have this in external packages.
comment:9 followup: 11 Changed 21 months ago by
Replying to ghrocky:
If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.
That would be great! Thanks!
comment:10 Changed 21 months ago by
Replying to ghmmatera:
Natively, mathics is able to produce asymptote and SVG graphics, with the first having much better support than the second. Then, to get a png we need to compile the asymptote code to produce the image. But to do that, we need to install a lot of dependencies. The other way around is to use matplotlib, but this requires a new implementation. This is why we have this in external packages.
I see! But, I think we can postpone the PNG
availability to a follow up ticket. This is not essential to start with the interface. At least, if you convert to Sage objects you can do plotting there.
comment:11 followup: 12 Changed 21 months ago by
Replying to soehms:
Replying to ghrocky:
If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.
That would be great! Thanks!
Ok tentatively then we can plan on a release the weekend of May 1516, assuming that Mauricio is on board with this.
comment:12 followup: 13 Changed 21 months ago by
Replying to ghrocky:
Replying to soehms:
Replying to ghrocky:
If it will help simplify things, (and personally I think it will), we can put out another release shortly. I generally would like to do so roughly every month that there are significant changes and I think in the last month there have been.
That would be great! Thanks!
Mathics3 2.2.0 has been released. Hopefully it addresses the problems with the last release. The next release should aslo add FindMinimum
.
comment:13 Changed 21 months ago by
Replying to ghrocky:
Mathics3 2.2.0 has been released. Hopefully it addresses the problems with the last release.
Thanks! These problems are solved. But on sytsems where lxml
is not present I'm getting a startup message lxml.html is not available...
, now, which distrubes the first doctest:
File "src/sage/interfaces/mathics.py", line 37, in sage.interfaces.mathics Failed example: mobj = mathics(x^21); mobj # optional  mathics Expected: 1 + x ^ 2 Got: lxml.html is not available... 1 + x ^ 2 ********************************************************************** 1 item had failures: 1 of 87 in sage.interfaces.mathics [222 tests, 1 failure, 9.45 s]
Is this something essential? If not: is there a way to supress this message in MathicsSession
? Or, is there a missing dependency?
comment:14 Changed 21 months ago by
This dependency was newly added to the last release when an HTML import feature was added.
It shouldn't be necessary. Perhaps down the line, loading various types of files will moved to packages outside of the Mathics core.
When https://github.com/mathics/Mathics/pull/1379 lands, it should address this by making the lxml
package optional. And next release we may have a FindMinimum
as well.
For now though, I don't know what to suggest. Sigh.
Also, after this gets into sage, we should figure out how to incorporate sage testing into a broader set of tests we do. (We need this for other packages as well.)
comment:15 followup: 16 Changed 21 months ago by
I just thought of some more possibilities.
 install
lxml
in sage. If sage already has something for parsing HTML, then maybe we can use that in the next release of Mathics.  If
lxml
is not installed, provide your ownlxml
import stub with submodulehtml
and aparse
function in it.
comment:16 Changed 21 months ago by
Replying to ghrocky:
When https://github.com/mathics/Mathics/pull/1379 lands, it should address this by making the lxml package optional. And next release we may have a FindMinimum? as well.
Thats fine!
install lxml in sage ...
If lxml
is optional in Mathics then it doesn't make sense to do anything in Sage (especially outside this interface) just to avoid this message. If a Sage user wants to use the optional functionality of Mathics via the interface then he can install it manually by sage pip install lxml
.
If sage already has something for parsing HTML, then maybe we can use that in the next release of Mathics.
AFAIK, html parsers that can be used per default in Sage are those of the following libraries:
sage: import html sage: import html5lib
comment:17 Changed 21 months ago by
Ok. We'll go with one of those, probably html
, which comes built in to Python in the version ranges we support, and most likely works just fine.
comment:18 Changed 21 months ago by
Commit:  85a3a0e5c3a0adf5bc03d9c417db3f5de68ac5d7 → 1051c6f82b6eb6a8aefebef9e6358f45d95f167a 

comment:19 Changed 21 months ago by
Authors:  → Sebastian Oehms 

Description:  modified (diff) 
With the current commit there are three expected doctest failures (one concerning lxml
and two with respect to FindMiminum
) which sould disappear with the next Mathics release.
comment:20 Changed 19 months ago by
Commit:  1051c6f82b6eb6a8aefebef9e6358f45d95f167a → cdd4328915640991c9d829659155da54271d3136 

comment:21 Changed 19 months ago by
The upgrade to version 3.1.0 fixes the lxml
issue, but not the one with FindMiminum
.
comment:22 Changed 19 months ago by
Description:  modified (diff) 

comment:23 Changed 19 months ago by
Commit:  cdd4328915640991c9d829659155da54271d3136 → a30242a2fb89676c454cf16d90340401bc0d52b8 

Branch pushed to git repo; I updated commit sha1. New commits:
a30242a  31778: dependency to mathics_scanner added

comment:24 Changed 18 months ago by
Milestone:  sage9.4 → sage9.5 

comment:25 Changed 17 months ago by
Commit:  a30242a2fb89676c454cf16d90340401bc0d52b8 → 5e68c59a618f29aa1d98a515360acbae7ebdb629 

Branch pushed to git repo; I updated commit sha1. New commits:
8cac399  Merge branch 'u/soehms/mathics_interface' of trac.sagemath.org:sage into mathics_interface_31778

1565c92  31778: update to Mathics3 4.0.0

a95d91c  31778: ignore FindMinimum doctests

5e68c59  31778: add dependency importlib_metadata (because of Python 3.7)

comment:26 Changed 17 months ago by
Status:  new → needs_review 

The commits above contain an update to the current release of Mathics3 introducing two new dependencies to pint
and palettable
. Still, the Mathematica function FindMinimum
is not present in this version. But anyway, I think this is not necessary for a start with the interface. Thus, I marked the corresponding doctests to be ignored.
I've started a workflow which is identical to toxoptional.yml
except that the matrix for the optional packages is reduced to mathics
.
It seems that there are no failures which are related to the new package except on debian buster one failing doctest on mathics.py
. This was caused by the missing dependency to importlib_metadata
which is needed for Sage on Python 3.7 and which is added in the current commit. Rerunning the debian buster test looks good now, as well.
Therefore, I think the ticket is ready for review, right now!
comment:27 Changed 17 months ago by
Description:  modified (diff) 

comment:28 followup: 29 Changed 17 months ago by
Reviewers:  → Dima Pasechnik 

Status:  needs_review → positive_review 
looks good to me, thanks
comment:29 Changed 17 months ago by
comment:30 Changed 17 months ago by
Branch:  u/soehms/mathics_interface → 5e68c59a618f29aa1d98a515360acbae7ebdb629 

Resolution:  → fixed 
Status:  positive_review → closed 
The initial version for the interface is an adaption of the corresponding interface for Mathematica. But in contrast it directly uses the Mathics library functionality instead of Pexpect. It contains all doctests inherited from the Mathematica module but also additional ones which are more Mathics specific. Furthermore, the coverage of doctests is complete.
Two of the Mathematica doctests use
FindMinimum
which doesn't seem to be available in Mathics. I mark them asnot tested
. 25 doctests had differences to the output that once has been produced by Mathematica. Most of them are caused by a different way of formatting. I've accepted them all, since they where either minor or a task for Mathics developers to decide if they are acceptable or not (see also this comment in Mathics issue 1235). Explicitly these are:Observe that the graphical support of Mathics is restricted to
SVG
. For the usage in Sage it would be preferable to havePNG
possible, as well, or to convert it here toPNG
(for example usingsvg2rlg
introducing a new dependency tosvglib
).Sage package integration is initialized using:
I rename the created package
Mathics3
tomathics
to match Sage package naming conventions.New commits:
29009: initial version
29009: add comment to doctest
Merge branch 'u/soehms/mpmath_divmod_29009' of trac.sagemath.org:sage into mathics_interface
31676: initial version
Merge branch 'u/soehms/mpmath_hash' of trac.sagemath.org:sage into mathics_interface
31775: initial version
Merge branch 'u/soehms/complex_interfaces_31775' of trac.sagemath.org:sage into mathics_interface
31778: initial version