Opened 3 years ago

Closed 21 months ago

#29497 closed enhancement (fixed)

package symengine and its Python interface

Reported by: Vincent Delecroix Owned by:
Priority: major Milestone: sage-9.3
Component: symbolics Keywords: thursdaysbdx, sd109
Cc: Isuru Fernando, Matthias Köppe, Eric Gourgoulhon Merged in:
Authors: Vincent Delecroix, Isuru Fernando, Matthias Koeppe Reviewers: Matthias Koeppe, Vincent Delecroix
Report Upstream: N/A Work issues:
Branch: d614ee2 (Commits, GitHub, GitLab) Commit: d614ee2dec3914a75e14fa79d7881da537faccb5
Dependencies: Stopgaps:

Status badges

Description (last modified by Matthias Köppe)

Symengine is a C++ symbolic engine

  • that can be used as a backend in sympy (and makes it much faster)
  • that implements "fast callable" for floats, GMP types, flint types, arb types, ... and accepts numpy arrays as arguments (the function is applied entrywise)

It is very strong candidate for replacing some of the symbolic we have in Sage, in particular all the fast callable.

See also:

Change History (70)

comment:1 Changed 3 years ago by Isuru Fernando

Let me know if you run into any trouble.

Here are some of the options I use in conda. https://github.com/conda-forge/symengine-feedstock/blob/master/recipe/build.sh#L6-L19. They should work fine for sage (except for WITH_LLVM=yes)

Dependencies: CMake, GMP/MPIR, MPFR, MPC, FLINT2, ARB

comment:2 Changed 3 years ago by Vincent Delecroix

Branch: public/29497
Commit: 69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060

New commits:

69e3c0229497: package symengine

comment:3 Changed 3 years ago by git

Commit: 69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060b5304c1531c5c8ede1cd32d792951f6eedf24393

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b5304c129497: package symengine

comment:4 Changed 3 years ago by Vincent Delecroix

Cc: Matthias Köppe added
Description: modified (diff)
Milestone: sage-9.2sage-9.1
Status: newneeds_review

comment:5 Changed 3 years ago by Matthias Köppe

Please add upstream_url to checksums.ini and distros/ Information

comment:6 Changed 3 years ago by Vincent Delecroix

I intentionally did write upstream_url because the tarballs are from github with their tags as names.

comment:7 Changed 3 years ago by git

Commit: b5304c1531c5c8ede1cd32d792951f6eedf24393a008f333c08881f762067c436c046bc361ad9fa0

Branch pushed to git repo; I updated commit sha1. New commits:

a008f33distros/conda.txt file

comment:8 Changed 3 years ago by Vincent Delecroix

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

comment:9 Changed 3 years ago by Isuru Fernando

symengine.py needs cmake as well.

comment:10 Changed 3 years ago by git

Commit: a008f333c08881f762067c436c046bc361ad9fa01a0b08631acd3c22a1dac32d4522f143d8bb4970

Branch pushed to git repo; I updated commit sha1. New commits:

1a0b086symengine.py also depends on cmake

comment:11 Changed 3 years ago by Isuru Fernando

Description: modified (diff)

comment:12 in reply to:  8 ; Changed 3 years ago by Matthias Köppe

Replying to vdelecroix:

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

The upstream_url code handles renaming just fine

  • build/pkgs/symengine.py/checksums.ini

    diff --git a/build/pkgs/symengine.py/checksums.ini b/build/pkgs/symengine.py/checksums.ini
    index ea4c44b468..489d040bcc 100644
    a b tarball=symengine.py-VERSION.tar.gz 
    22sha1=40df2b8f406b6ac4a86c83a82d31593b5738a470
    33md5=52b035da7851414d74f3bde83b6c2976
    44cksum=2829876300
     5upstream_url=https://github.com/symengine/symengine.py/archive/vVERSION.tar.gz
  • build/pkgs/symengine/checksums.ini

    diff --git a/build/pkgs/symengine/checksums.ini b/build/pkgs/symengine/checksums.ini
    index 8143c8eae5..b2651f694c 100644
    a b tarball=symengine-VERSION.tar.gz 
    22sha1=abd7d39b1b724f47bcdc5b1d811cf33f2c413aea
    33md5=3c0df2b14310467c6d45bc26a557324b
    44cksum=100973134
     5upstream_url=https://github.com/symengine/symengine/archive/vVERSION.tar.gz

comment:13 in reply to:  12 Changed 3 years ago by Vincent Delecroix

Replying to mkoeppe:

Replying to vdelecroix:

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

The upstream_url code handles renaming just fine

Wunderbar!

comment:14 Changed 3 years ago by git

Commit: 1a0b08631acd3c22a1dac32d4522f143d8bb4970fdf87a00b447ec768de7b8b6db1709588701bd3c

Branch pushed to git repo; I updated commit sha1. New commits:

fdf87a0upstream_url

comment:15 Changed 3 years ago by Matthias Köppe

Reviewers: Matthias Koeppe
Status: needs_reviewpositive_review

Builds and tests OK on macOS with python3. Could you add a little bit to the ticket description please - what is it intended to be used for in Sage

comment:16 Changed 3 years ago by Isuru Fernando

Authors: Vincent DelecroixVincent Delecroix, Isuru Fernando
Branch: public/29497u/isuruf/29497
Commit: fdf87a00b447ec768de7b8b6db1709588701bd3c42d49e164f7ee43678ce3ec393bd37d429f9b29a
Status: positive_reviewneeds_review

New commits:

dcf980aMake sure symengine finds dependencies from SAGE_LOCAL
42d49e1Fix testing

comment:17 Changed 3 years ago by Isuru Fernando

'import symengine; symengine.test()' uses pytest. Switched to nose

comment:18 Changed 3 years ago by Vincent Delecroix

Description: modified (diff)

comment:19 Changed 3 years ago by Vincent Delecroix

As tests are not run by default, I don't like this so much

-symengine $(PYTHON) | cmake pip setuptools
+symengine $(PYTHON) | cmake pip setuptools nose

@Matthias: how do we solve that when tests depend on additional libraries? (same thing actually hapenning with numpy, rpy2, etc)

comment:20 Changed 3 years ago by Matthias Köppe

Something like this (untested)?

  • build/pkgs/symengine.py/dependencies

    a b  
    1 symengine $(PYTHON) | cmake pip setuptools
     1symengine $(PYTHON) | cmake pip setuptools $(and $(filter-out no,$(SAGE_CHECK)), nose)
    22
    33----------
    44All lines of this file are ignored except the first.
  • src/bin/sage

    diff --git a/src/bin/sage b/src/bin/sage
    index 10acddcd96..2106102952 100755
    a b if [ "$1" = '-i' ]; then 
    359359                echo >&2 "Error: 'sage -i $OPT <package>' is no longer supported, use 'sage --info <package>' instead."
    360360                exit 2;;
    361361            -f) FORCE_INSTALL=yes;;
     362            # Setting SAGE_CHECK here duplicates what we do in sage-spkg
     363            # but we need it in "make" already when there are (order-only)
     364            # dependencies on packages providing test infrastructure
     365            -c) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=yes;;
     366            -w) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=warn;;
    362367            -*) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT";;
    363368            *) PACKAGES="$PACKAGES $OPT";;
    364369        esac

comment:21 Changed 3 years ago by Matthias Köppe

Forgot export SAGE_CHECK obviously

comment:22 Changed 3 years ago by Matthias Köppe

Milestone: sage-9.1sage-9.2

comment:23 Changed 3 years ago by Vincent Delecroix

Status: needs_reviewneeds_work

The doctester does not like dot

Traceback (most recent call last):
  File "/opt/sage/src/bin/sage-runtests", line 177, in <module>
    DC = DocTestController(options, args)
  File "/opt/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 365, in __init__
    raise ValueError('invalid optional tag {!r}'.format(o))
ValueError: invalid optional tag 'symengine.py'

comment:24 Changed 3 years ago by Vincent Delecroix

Branch: u/isuruf/29497public/29497
Commit: 42d49e164f7ee43678ce3ec393bd37d429f9b29a1e57389528ff32491b637daef08cf2915b7e5c2c
Status: needs_workneeds_review

New commits:

f8154a529497: package symengine
c5b7d97distros/conda.txt file
cb0b173symengine.py also depends on cmake
4fa6c1fupstream_url
34593abchange names symengine.py -> symengine_py
9dbca0bMake sure symengine finds dependencies from SAGE_LOCAL
1e57389Fix testing

comment:25 Changed 3 years ago by git

Commit: 1e57389528ff32491b637daef08cf2915b7e5c2c6ef49e056b3f52587cfbf0fa38b252b01e09d42a

Branch pushed to git repo; I updated commit sha1. New commits:

6ef49e0a simple example

comment:26 Changed 3 years ago by Vincent Delecroix

@isuruf: I think this ticket is ready for inclusion. Next step is I believe to implement the conversion Sage -> symengine (which is not there, right?)

comment:27 Changed 3 years ago by Vincent Delecroix

Keywords: thursdaysbdx added

comment:28 Changed 3 years ago by Matthias Köppe

Description: modified (diff)

comment:29 Changed 3 years ago by Matthias Köppe

Does sympy have to be configured specifically to use this package? Does the package have to be available at installation time of sympy?

comment:30 Changed 3 years ago by Isuru Fernando

symengine and sympy can interoperate without any configuration.

There's a feature in sympy enabled by the env variable USE_SYMENGINE=1 at runtime which will make some modules of sympy use symengine as the core symbolic engine. This is still experimental work.

comment:31 Changed 3 years ago by Ondřej Čertík

@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine.

Last edited 3 years ago by Ondřej Čertík (previous) (diff)

comment:32 Changed 3 years ago by Matthias Köppe

Status: needs_reviewneeds_work
[symengine_py-0.6.1] Installing symengine_py-0.6.1
[symengine_py-0.6.1] /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/bin/sage-pip-install requires . as final argument

comment:33 Changed 3 years ago by git

Commit: 6ef49e056b3f52587cfbf0fa38b252b01e09d42abb7c6c2d0686930bd02100c746ce808250ba8ab2

Branch pushed to git repo; I updated commit sha1. New commits:

bb7c6c2Merge 9.2.beta0

comment:34 Changed 3 years ago by git

Commit: bb7c6c2d0686930bd02100c746ce808250ba8ab230fce9ca0d71ec31621f82fc77eb5c380b4bcba0

Branch pushed to git repo; I updated commit sha1. New commits:

30fce9cmove . at the end in sdh_pip_install

comment:35 Changed 3 years ago by Vincent Delecroix

Status: needs_workneeds_review

comment:36 Changed 3 years ago by git

Commit: 30fce9ca0d71ec31621f82fc77eb5c380b4bcba02d0a81fe236ff03f3d90b8bcbf8398adaecc8865

Branch pushed to git repo; I updated commit sha1. New commits:

2d0a81f.github/workflows: Add symengine_py

comment:38 Changed 3 years ago by Matthias Köppe

Keywords: sd109 added

comment:39 Changed 3 years ago by Matthias Köppe

Status: needs_reviewneeds_work

symengine_py build fails:

    running build_ext
    error: error in command line: command 'BuildExtWithCmake' has no such option 'no_user_cfg'
    Running setup.py install for symengine: finished with status 'error'

(see https://github.com/mkoeppe/sage/runs/728445828?check_suite_focus=true)

comment:40 Changed 2 years ago by git

Commit: 2d0a81fe236ff03f3d90b8bcbf8398adaecc88655040a042aedae753af6b35f2b88ef323393f85d0

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

178e0c529497: package symengine
3da3b6fdistros/conda.txt file
f819ee9symengine.py also depends on cmake
aa83ce3upstream_url
950f88dchange names symengine.py -> symengine_py
4b3bba8Make sure symengine finds dependencies from SAGE_LOCAL
45b60c7Fix testing
898ed4fa simple example
a331b70move . at the end in sdh_pip_install
5040a04.github/workflows: Add symengine_py

comment:41 Changed 2 years ago by Matthias Köppe

Rebased on 9.2.beta3

comment:42 Changed 2 years ago by git

Commit: 5040a042aedae753af6b35f2b88ef323393f85d03bc107bde4dc8d0ee3839b0eeb23a5e4c672b373

Branch pushed to git repo; I updated commit sha1. New commits:

3bc107bbuild/pkgs/symengine_py/spkg-install.in: Remove bad args to pip install

comment:43 Changed 2 years ago by Matthias Köppe

comment:44 Changed 2 years ago by Matthias Köppe

Authors: Vincent Delecroix, Isuru FernandoVincent Delecroix, Isuru Fernando, Matthias Koeppe
Status: needs_workneeds_review

comment:45 Changed 2 years ago by Vincent Delecroix

Maybe

-upstream_url=https://github.com/symengine/symengine/releases/download/v0.6.0/symengine-VERSION.tar.gz
+upstream_url=https://github.com/symengine/symengine/releases/download/vVERSION/symengine-VERSION.tar.gz

comment:46 Changed 2 years ago by git

Commit: 3bc107bde4dc8d0ee3839b0eeb23a5e4c672b3736da2383db12ed3fd2b49ab014c2737865b3c1de8

Branch pushed to git repo; I updated commit sha1. New commits:

6da2383build/pkgs/symengine/checksums.ini: More version templating

comment:47 in reply to:  43 Changed 2 years ago by Matthias Köppe

Replying to mkoeppe:

Tests run at https://github.com/mkoeppe/sage/actions/runs/157858355

The docker-based tests didn't go through because they are broken in 9.2.beta3 (#30064 has the fix).

But the cygwin-standard build (https://github.com/mkoeppe/sage/runs/838016220) reveals a build problem:

[arb-2.16.0.p0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/arb-2.16.0.p0.log
[ecm-7.0.4.p1] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/ecm-7.0.4.p1.log
  [ecm-7.0.4.p1] successfully installed.
  [arb-2.16.0.p0] successfully installed.
sage-logger -p 'sage-spkg -y -o  symengine-0.6.0' '/cygdrive/d/a/sage/sage/logs/pkgs/symengine-0.6.0.log'
[symengine-0.6.0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/symengine-0.6.0.log
  [symengine-0.6.0] error installing, exit status 1. End of log file:
  [symengine-0.6.0]   -- Check size of long double
  [symengine-0.6.0]   -- Check size of long double - done
  [symengine-0.6.0]   -- Found GMP: /usr/lib/libgmp.dll.a  
  [symengine-0.6.0]   -- Found ECM: /cygdrive/d/a/sage/sage/local/lib/libecm.dll.a  
  [symengine-0.6.0]   -- Found FLINT: /usr/lib/libflint.dll.a  
  [symengine-0.6.0]   CMake Error at /usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  [symengine-0.6.0]     Could NOT find ARB (missing: ARB_LIBRARIES)
  [symengine-0.6.0]   Call Stack (most recent call first):
  [symengine-0.6.0]     /usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  [symengine-0.6.0]     cmake/FindARB.cmake:11 (find_package_handle_standard_args)
  [symengine-0.6.0]     CMakeLists.txt:336 (find_package)
  [symengine-0.6.0]   
  [symengine-0.6.0]   
  [symengine-0.6.0]   -- Configuring incomplete, errors occurred!

comment:48 Changed 2 years ago by Matthias Köppe

Status: needs_reviewneeds_work

comment:49 Changed 2 years ago by Isuru Fernando

This is an issue with arb not installing the import library on windows. https://github.com/fredrik-johansson/arb/pull/315 should fix it

comment:50 Changed 2 years ago by git

Commit: 6da2383db12ed3fd2b49ab014c2737865b3c1de8055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b

Branch pushed to git repo; I updated commit sha1. New commits:

539c182build/make/install: Do not depend on src/bin/sage-version.sh
761092cMerge branch 't/29987/build_make_install__do_not_depend_on_src_bin_sage_version_sh' into t/30064/fix_tox_docker_builds_broken_by__29884
f2efa6asrc/doc/bootstrap: Create the directory src/doc/en/reference/repl if it does not exist
b7bf43bbuild/bin/write-dockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884
365ce61Merge branch 'u/mkoeppe/fix_tox_docker_builds_broken_by__29884' of git://trac.sagemath.org/sage into HEAD
1e7becctox.ini [debian-buster, -sid]: IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7-dev
fb61a31Merge branch 'u/mkoeppe/tox_ini__debian_bullseye___sid_have_python3_8_instead_of_3_7' of git://trac.sagemath.org/sage into 9.2.beta3+ci-fixes
055b6b4Merge branch '9.2.beta3+ci-fixes' into t/29497/public/29497

comment:51 in reply to:  49 Changed 2 years ago by Matthias Köppe

Replying to isuruf:

This is an issue with arb not installing the import library on windows. https://github.com/fredrik-johansson/arb/pull/315 should fix it

It's marked as a draft - should we be testing it here on the ticket?

comment:52 Changed 2 years ago by git

Commit: 055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b53f960de9f0e8f83c3c89033ed5fef7d8ff28c42

Branch pushed to git repo; I updated commit sha1. New commits:

53f960dMerge tag '9.2.beta4' into t/29497/public/29497

comment:53 Changed 2 years ago by git

Commit: 53f960de9f0e8f83c3c89033ed5fef7d8ff28c4279b0f3af36ec4269c367d5b0acb9afd9bb602949

Branch pushed to git repo; I updated commit sha1. New commits:

bfb2fb4tox.ini: Add environment local-homebrew-usrlocal
aded967Merge commit '8195cb821e01e0cb627240d7135d5634c265cab0' of git://trac.sagemath.org/sage into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
07d657dMerge tag '9.2.beta4' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
bd15fb9build/bin/write-dockerfile.sh, tox.ini: Do not run testsuites that need pip (ssl)
79b0f3aMerge branch 't/29929/tox_ini__add_a_macos_environment_without_homebrew__conda' into t/29497/public/29497

comment:54 Changed 2 years ago by Matthias Köppe

Dependencies: #29929

comment:55 Changed 2 years ago by Matthias Köppe

Dependencies: #29929#29929, #30118

comment:56 Changed 2 years ago by git

Commit: 79b0f3af36ec4269c367d5b0acb9afd9bb602949b63ac91d9607ead738829db1065f527b84f9128e

Branch pushed to git repo; I updated commit sha1. New commits:

25393b0Handle SAGE_CHECK_PACKAGES in build/make/Makefile.in, not sage-spkg
e4ec94bMerge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29497/public/29497
b63ac91build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:57 Changed 2 years ago by Matthias Köppe

Milestone: sage-9.2sage-9.3

comment:58 in reply to:  31 Changed 23 months ago by Dima Pasechnik

Replying to certik:

@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine.

How about Documentation? :-)

comment:59 Changed 23 months ago by Siddharth

@dimpase: What kind of documentation would help? I'm also interested in pushing this along!

comment:60 Changed 23 months ago by git

Commit: b63ac91d9607ead738829db1065f527b84f9128e2a1566da3637703f280d4dc2ab35fcf53abbc977

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

ed8a041symengine.py also depends on cmake
b1aade0upstream_url
08b96d2change names symengine.py -> symengine_py
595983eMake sure symengine finds dependencies from SAGE_LOCAL
c60012fFix testing
2f863eaa simple example
7071c14move . at the end in sdh_pip_install
e157f84build/pkgs/symengine_py/spkg-install.in: Remove bad args to pip install
ec1d25cbuild/pkgs/symengine/checksums.ini: More version templating
2a1566dbuild/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:61 Changed 23 months ago by Matthias Köppe

Rebased on current develop

comment:62 Changed 23 months ago by Matthias Köppe

Dependencies: #29929, #30118

comment:63 Changed 23 months ago by Matthias Köppe

Description: modified (diff)

comment:64 Changed 23 months ago by git

Commit: 2a1566da3637703f280d4dc2ab35fcf53abbc977d614ee2dec3914a75e14fa79d7881da537faccb5

Branch pushed to git repo; I updated commit sha1. New commits:

983bcc7build/pkgs/symengine*: Change SPKG.txt to SPKG.rst
4d26ab4build/pkgs/symengine_py/install-requires.txt: New
cba3613build/pkgs/symengine_py/dependencies: Update
985e7b1build/pkgs/symengine/distros: Add more
d614ee2build/pkgs/symengine_py/distros/repology.txt: New

comment:65 Changed 23 months ago by Matthias Köppe

Reviewers: Matthias KoeppeMatthias Koeppe, ...
Status: needs_workneeds_review

I've updated the package metadata. Ready for review.

comment:66 Changed 23 months ago by Eric Gourgoulhon

Cc: Eric Gourgoulhon added

comment:67 Changed 23 months ago by Matthias Köppe

Description: modified (diff)

comment:68 Changed 22 months ago by Vincent Delecroix

Reviewers: Matthias Koeppe, ...Matthias Koeppe, Vincent Delecroix
Status: needs_reviewpositive_review

Install worked fine and tests pass! Time to move forward.

comment:69 Changed 22 months ago by Matthias Köppe

Great, thanks.

comment:70 Changed 21 months ago by Volker Braun

Branch: public/29497d614ee2dec3914a75e14fa79d7881da537faccb5
Resolution: fixed
Status: positive_reviewclosed
Note: See TracTickets for help on using tickets.