package symengine and its Python interface

[Vincent Delecroix]

Symengine is a C++ symbolic engine

• that can be used as a backend in sympy (and makes it much faster)
• that implements "fast callable" for floats, GMP types, flint types, arb types, ... and accepts numpy arrays as arguments (the function is applied entrywise)

It is very strong candidate for replacing some of the symbolic we have in Sage, in particular all the fast callable.

• ​https://github.com/symengine/symengine/releases
• ​https://github.com/symengine/symengine.py/

See also:

• ​https://groups.google.com/d/msg/sage-devel/m8XXaoIpvIw/Eeju5I61CQAJ (discussion August 2015)
• ​https://groups.google.com/d/msg/sympy/bPh2Hx1HRJ4/CLyuv2c1AgAJ (discussion May 2020)
• ​https://groups.google.com/g/sage-devel/c/sY3zh-pq8T4/m/Y2Bp0_LxAAAJ (discussion January 2021)

[Isuru Fernando]

Let me know if you run into any trouble.

Here are some of the options I use in conda. ​https://github.com/conda-forge/symengine-feedstock/blob/master/recipe/build.sh#L6-L19. They should work fine for sage (except for WITH_LLVM=yes)

Dependencies: CMake, GMP/MPIR, MPFR, MPC, FLINT2, ARB

[Vincent Delecroix]

New commits:

​69e3c02

29497: package symengine

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b5304c1

29497: package symengine

[Matthias Köppe]

Please add upstream_url to checksums.ini and distros/ Information

[Vincent Delecroix]

I intentionally did write upstream_url because the tarballs are from github with their tags as names.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​a008f33

distros/conda.txt file

[Vincent Delecroix]

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

[Isuru Fernando]

symengine.py needs cmake as well.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​1a0b086

symengine.py also depends on cmake

[Matthias Köppe]

Replying to vdelecroix:

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

The upstream_url code handles renaming just fine

build/pkgs/symengine.py/checksums.ini

@@ -2 +2 @@
 sha1=40df2b8f406b6ac4a86c83a82d31593b5738a470
 md5=52b035da7851414d74f3bde83b6c2976
 cksum=2829876300
+upstream_url=https://github.com/symengine/symengine.py/archive/vVERSION.tar.gz

build/pkgs/symengine/checksums.ini

@@ -2 +2 @@
 sha1=abd7d39b1b724f47bcdc5b1d811cf33f2c413aea
 md5=3c0df2b14310467c6d45bc26a557324b
 cksum=100973134
+upstream_url=https://github.com/symengine/symengine/archive/vVERSION.tar.gz

[Vincent Delecroix]

Replying to mkoeppe:

Replying to vdelecroix:

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

The upstream_url code handles renaming just fine

Wunderbar!

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fdf87a0

upstream_url

[Matthias Köppe]

Builds and tests OK on macOS with python3. Could you add a little bit to the ticket description please - what is it intended to be used for in Sage

[Isuru Fernando]

New commits:

​dcf980a	Make sure symengine finds dependencies from SAGE_LOCAL
​42d49e1	Fix testing

[Isuru Fernando]

'import symengine; symengine.test()' uses pytest. Switched to nose

[Vincent Delecroix]

As tests are not run by default, I don't like this so much

-symengine $(PYTHON) | cmake pip setuptools
+symengine $(PYTHON) | cmake pip setuptools nose

@Matthias: how do we solve that when tests depend on additional libraries? (same thing actually hapenning with numpy, rpy2, etc)

[Matthias Köppe]

Something like this (untested)?

build/pkgs/symengine.py/dependencies

@@ -1 @@
-symengine $(PYTHON) | cmake pip setuptools
+symengine $(PYTHON) | cmake pip setuptools $(and $(filter-out no,$(SAGE_CHECK)), nose)
 
 ----------
 All lines of this file are ignored except the first.

src/bin/sage

@@ -359 +359 @@
                 echo >&2 "Error: 'sage -i $OPT <package>' is no longer supported, use 'sage --info <package>' instead."
                 exit 2;;
             -f) FORCE_INSTALL=yes;;
+            # Setting SAGE_CHECK here duplicates what we do in sage-spkg
+            # but we need it in "make" already when there are (order-only)
+            # dependencies on packages providing test infrastructure
+            -c) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=yes;;
+            -w) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=warn;;
             -*) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT";;
             *) PACKAGES="$PACKAGES $OPT";;
         esac

[Matthias Köppe]

Forgot export SAGE_CHECK obviously

[Vincent Delecroix]

The doctester does not like dot

Traceback (most recent call last):
  File "/opt/sage/src/bin/sage-runtests", line 177, in <module>
    DC = DocTestController(options, args)
  File "/opt/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 365, in __init__
    raise ValueError('invalid optional tag {!r}'.format(o))
ValueError: invalid optional tag 'symengine.py'

[Vincent Delecroix]

New commits:

​f8154a5	29497: package symengine
​c5b7d97	distros/conda.txt file
​cb0b173	symengine.py also depends on cmake
​4fa6c1f	upstream_url
​34593ab	change names symengine.py -> symengine_py
​9dbca0b	Make sure symengine finds dependencies from SAGE_LOCAL
​1e57389	Fix testing

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6ef49e0

a simple example

[Vincent Delecroix]

@isuruf: I think this ticket is ready for inclusion. Next step is I believe to implement the conversion Sage -> symengine (which is not there, right?)

[Matthias Köppe]

Does sympy have to be configured specifically to use this package? Does the package have to be available at installation time of sympy?

[Isuru Fernando]

symengine and sympy can interoperate without any configuration.

There's a feature in sympy enabled by the env variable USE_SYMENGINE=1 at runtime which will make some modules of sympy use symengine as the core symbolic engine. This is still experimental work.

[Ondřej Čertík]

@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine.

[Matthias Köppe]

[symengine_py-0.6.1] Installing symengine_py-0.6.1
[symengine_py-0.6.1] /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/bin/sage-pip-install requires . as final argument

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bb7c6c2

Merge 9.2.beta0

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​30fce9c

move . at the end in sdh_pip_install

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2d0a81f

.github/workflows: Add symengine_py

[Matthias Köppe]

Tests running at ​https://github.com/mkoeppe/sage/actions/runs/121775190

[Matthias Köppe]

symengine_py build fails:

    running build_ext
    error: error in command line: command 'BuildExtWithCmake' has no such option 'no_user_cfg'
    Running setup.py install for symengine: finished with status 'error'

(see ​https://github.com/mkoeppe/sage/runs/728445828?check_suite_focus=true)

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​178e0c5	29497: package symengine
​3da3b6f	distros/conda.txt file
​f819ee9	symengine.py also depends on cmake
​aa83ce3	upstream_url
​950f88d	change names symengine.py -> symengine_py
​4b3bba8	Make sure symengine finds dependencies from SAGE_LOCAL
​45b60c7	Fix testing
​898ed4f	a simple example
​a331b70	move . at the end in sdh_pip_install
​5040a04	.github/workflows: Add symengine_py

[Matthias Köppe]

Rebased on 9.2.beta3

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​3bc107b

build/pkgs/symengine_py/spkg-install.in: Remove bad args to pip install

[Matthias Köppe]

Tests run at ​https://github.com/mkoeppe/sage/actions/runs/157858355

[Vincent Delecroix]

Maybe

-upstream_url=https://github.com/symengine/symengine/releases/download/v0.6.0/symengine-VERSION.tar.gz
+upstream_url=https://github.com/symengine/symengine/releases/download/vVERSION/symengine-VERSION.tar.gz

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6da2383

build/pkgs/symengine/checksums.ini: More version templating

[Matthias Köppe]

Replying to mkoeppe:

Tests run at ​https://github.com/mkoeppe/sage/actions/runs/157858355

The docker-based tests didn't go through because they are broken in 9.2.beta3 (#30064 has the fix).

But the cygwin-standard build (​https://github.com/mkoeppe/sage/runs/838016220) reveals a build problem:

[arb-2.16.0.p0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/arb-2.16.0.p0.log
[ecm-7.0.4.p1] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/ecm-7.0.4.p1.log
  [ecm-7.0.4.p1] successfully installed.
  [arb-2.16.0.p0] successfully installed.
sage-logger -p 'sage-spkg -y -o  symengine-0.6.0' '/cygdrive/d/a/sage/sage/logs/pkgs/symengine-0.6.0.log'
[symengine-0.6.0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/symengine-0.6.0.log
  [symengine-0.6.0] error installing, exit status 1. End of log file:
  [symengine-0.6.0]   -- Check size of long double
  [symengine-0.6.0]   -- Check size of long double - done
  [symengine-0.6.0]   -- Found GMP: /usr/lib/libgmp.dll.a  
  [symengine-0.6.0]   -- Found ECM: /cygdrive/d/a/sage/sage/local/lib/libecm.dll.a  
  [symengine-0.6.0]   -- Found FLINT: /usr/lib/libflint.dll.a  
  [symengine-0.6.0]   CMake Error at /usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  [symengine-0.6.0]     Could NOT find ARB (missing: ARB_LIBRARIES)
  [symengine-0.6.0]   Call Stack (most recent call first):
  [symengine-0.6.0]     /usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  [symengine-0.6.0]     cmake/FindARB.cmake:11 (find_package_handle_standard_args)
  [symengine-0.6.0]     CMakeLists.txt:336 (find_package)
  [symengine-0.6.0]   
  [symengine-0.6.0]   
  [symengine-0.6.0]   -- Configuring incomplete, errors occurred!

[Isuru Fernando]

This is an issue with arb not installing the import library on windows. ​https://github.com/fredrik-johansson/arb/pull/315 should fix it

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​539c182	build/make/install: Do not depend on src/bin/sage-version.sh
​761092c	Merge branch 't/29987/build_make_install__do_not_depend_on_src_bin_sage_version_sh' into t/30064/fix_tox_docker_builds_broken_by__29884
​f2efa6a	src/doc/bootstrap: Create the directory src/doc/en/reference/repl if it does not exist
​b7bf43b	build/bin/write-dockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884
​365ce61	Merge branch 'u/mkoeppe/fix_tox_docker_builds_broken_by__29884' of git://trac.sagemath.org/sage into HEAD
​1e7becc	tox.ini [debian-buster, -sid]: IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7-dev
​fb61a31	Merge branch 'u/mkoeppe/tox_ini__debian_bullseye___sid_have_python3_8_instead_of_3_7' of git://trac.sagemath.org/sage into 9.2.beta3+ci-fixes
​055b6b4	Merge branch '9.2.beta3+ci-fixes' into t/29497/public/29497

[Matthias Köppe]

Replying to isuruf:

This is an issue with arb not installing the import library on windows. ​https://github.com/fredrik-johansson/arb/pull/315 should fix it

It's marked as a draft - should we be testing it here on the ticket?

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​53f960d

Merge tag '9.2.beta4' into t/29497/public/29497

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bfb2fb4	tox.ini: Add environment local-homebrew-usrlocal
​aded967	Merge commit '8195cb821e01e0cb627240d7135d5634c265cab0' of git://trac.sagemath.org/sage into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
​07d657d	Merge tag '9.2.beta4' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
​bd15fb9	build/bin/write-dockerfile.sh, tox.ini: Do not run testsuites that need pip (ssl)
​79b0f3a	Merge branch 't/29929/tox_ini__add_a_macos_environment_without_homebrew__conda' into t/29497/public/29497

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​25393b0	Handle SAGE_CHECK_PACKAGES in build/make/Makefile.in, not sage-spkg
​e4ec94b	Merge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29497/public/29497
​b63ac91	build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

[Dima Pasechnik]

Replying to certik:

@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine.

How about Documentation? :-)

[Siddharth]

@dimpase: What kind of documentation would help? I'm also interested in pushing this along!

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

​ed8a041	symengine.py also depends on cmake
​b1aade0	upstream_url
​08b96d2	change names symengine.py -> symengine_py
​595983e	Make sure symengine finds dependencies from SAGE_LOCAL
​c60012f	Fix testing
​2f863ea	a simple example
​7071c14	move . at the end in sdh_pip_install
​e157f84	build/pkgs/symengine_py/spkg-install.in: Remove bad args to pip install
​ec1d25c	build/pkgs/symengine/checksums.ini: More version templating
​2a1566d	build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

[Matthias Köppe]

Rebased on current develop

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​983bcc7	build/pkgs/symengine*: Change SPKG.txt to SPKG.rst
​4d26ab4	build/pkgs/symengine_py/install-requires.txt: New
​cba3613	build/pkgs/symengine_py/dependencies: Update
​985e7b1	build/pkgs/symengine/distros: Add more
​d614ee2	build/pkgs/symengine_py/distros/repology.txt: New

[Matthias Köppe]

I've updated the package metadata. Ready for review.

[Vincent Delecroix]

Install worked fine and tests pass! Time to move forward.

[Matthias Köppe]

Great, thanks.