Opened 14 months ago

Closed 4 months ago

#29497 closed enhancement (fixed)

package symengine and its Python interface

Reported by: vdelecroix Owned by:
Priority: major Milestone: sage-9.3
Component: symbolics Keywords: thursdaysbdx, sd109
Cc: isuruf, mkoeppe, egourgoulhon Merged in:
Authors: Vincent Delecroix, Isuru Fernando, Matthias Koeppe Reviewers: Matthias Koeppe, Vincent Delecroix
Report Upstream: N/A Work issues:
Branch: d614ee2 (Commits, GitHub, GitLab) Commit: d614ee2dec3914a75e14fa79d7881da537faccb5
Dependencies: Stopgaps:

Status badges

Description (last modified by mkoeppe)

Symengine is a C++ symbolic engine

  • that can be used as a backend in sympy (and makes it much faster)
  • that implements "fast callable" for floats, GMP types, flint types, arb types, ... and accepts numpy arrays as arguments (the function is applied entrywise)

It is very strong candidate for replacing some of the symbolic we have in Sage, in particular all the fast callable.

See also:

Change History (70)

comment:1 Changed 14 months ago by isuruf

Let me know if you run into any trouble.

Here are some of the options I use in conda. https://github.com/conda-forge/symengine-feedstock/blob/master/recipe/build.sh#L6-L19. They should work fine for sage (except for WITH_LLVM=yes)

Dependencies: CMake, GMP/MPIR, MPFR, MPC, FLINT2, ARB

comment:2 Changed 14 months ago by vdelecroix

  • Branch set to public/29497
  • Commit set to 69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060

New commits:

69e3c0229497: package symengine

comment:3 Changed 14 months ago by git

  • Commit changed from 69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060 to b5304c1531c5c8ede1cd32d792951f6eedf24393

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

b5304c129497: package symengine

comment:4 Changed 14 months ago by vdelecroix

  • Cc mkoeppe added
  • Description modified (diff)
  • Milestone changed from sage-9.2 to sage-9.1
  • Status changed from new to needs_review

comment:5 Changed 14 months ago by mkoeppe

Please add upstream_url to checksums.ini and distros/ Information

comment:6 Changed 14 months ago by vdelecroix

I intentionally did write upstream_url because the tarballs are from github with their tags as names.

comment:7 Changed 14 months ago by git

  • Commit changed from b5304c1531c5c8ede1cd32d792951f6eedf24393 to a008f333c08881f762067c436c046bc361ad9fa0

Branch pushed to git repo; I updated commit sha1. New commits:

a008f33distros/conda.txt file

comment:8 follow-up: Changed 14 months ago by vdelecroix

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

comment:9 Changed 14 months ago by isuruf

symengine.py needs cmake as well.

comment:10 Changed 14 months ago by git

  • Commit changed from a008f333c08881f762067c436c046bc361ad9fa0 to 1a0b08631acd3c22a1dac32d4522f143d8bb4970

Branch pushed to git repo; I updated commit sha1. New commits:

1a0b086symengine.py also depends on cmake

comment:11 Changed 14 months ago by isuruf

  • Description modified (diff)

comment:12 in reply to: ↑ 8 ; follow-up: Changed 14 months ago by mkoeppe

Replying to vdelecroix:

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

The upstream_url code handles renaming just fine

  • build/pkgs/symengine.py/checksums.ini

    diff --git a/build/pkgs/symengine.py/checksums.ini b/build/pkgs/symengine.py/checksums.ini
    index ea4c44b468..489d040bcc 100644
    a b tarball=symengine.py-VERSION.tar.gz 
    22sha1=40df2b8f406b6ac4a86c83a82d31593b5738a470
    33md5=52b035da7851414d74f3bde83b6c2976
    44cksum=2829876300
     5upstream_url=https://github.com/symengine/symengine.py/archive/vVERSION.tar.gz
  • build/pkgs/symengine/checksums.ini

    diff --git a/build/pkgs/symengine/checksums.ini b/build/pkgs/symengine/checksums.ini
    index 8143c8eae5..b2651f694c 100644
    a b tarball=symengine-VERSION.tar.gz 
    22sha1=abd7d39b1b724f47bcdc5b1d811cf33f2c413aea
    33md5=3c0df2b14310467c6d45bc26a557324b
    44cksum=100973134
     5upstream_url=https://github.com/symengine/symengine/archive/vVERSION.tar.gz

comment:13 in reply to: ↑ 12 Changed 14 months ago by vdelecroix

Replying to mkoeppe:

Replying to vdelecroix:

(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine-0.6.1.tar.gz which collides with symengine tarballs)

The upstream_url code handles renaming just fine

Wunderbar!

comment:14 Changed 14 months ago by git

  • Commit changed from 1a0b08631acd3c22a1dac32d4522f143d8bb4970 to fdf87a00b447ec768de7b8b6db1709588701bd3c

Branch pushed to git repo; I updated commit sha1. New commits:

fdf87a0upstream_url

comment:15 Changed 14 months ago by mkoeppe

  • Reviewers set to Matthias Koeppe
  • Status changed from needs_review to positive_review

Builds and tests OK on macOS with python3. Could you add a little bit to the ticket description please - what is it intended to be used for in Sage

comment:16 Changed 14 months ago by isuruf

  • Authors changed from Vincent Delecroix to Vincent Delecroix, Isuru Fernando
  • Branch changed from public/29497 to u/isuruf/29497
  • Commit changed from fdf87a00b447ec768de7b8b6db1709588701bd3c to 42d49e164f7ee43678ce3ec393bd37d429f9b29a
  • Status changed from positive_review to needs_review

New commits:

dcf980aMake sure symengine finds dependencies from SAGE_LOCAL
42d49e1Fix testing

comment:17 Changed 14 months ago by isuruf

'import symengine; symengine.test()' uses pytest. Switched to nose

comment:18 Changed 14 months ago by vdelecroix

  • Description modified (diff)

comment:19 Changed 14 months ago by vdelecroix

As tests are not run by default, I don't like this so much

-symengine $(PYTHON) | cmake pip setuptools
+symengine $(PYTHON) | cmake pip setuptools nose

@Matthias: how do we solve that when tests depend on additional libraries? (same thing actually hapenning with numpy, rpy2, etc)

comment:20 Changed 14 months ago by mkoeppe

Something like this (untested)?

  • build/pkgs/symengine.py/dependencies

    a b  
    1 symengine $(PYTHON) | cmake pip setuptools
     1symengine $(PYTHON) | cmake pip setuptools $(and $(filter-out no,$(SAGE_CHECK)), nose)
    22
    33----------
    44All lines of this file are ignored except the first.
  • src/bin/sage

    diff --git a/src/bin/sage b/src/bin/sage
    index 10acddcd96..2106102952 100755
    a b if [ "$1" = '-i' ]; then 
    359359                echo >&2 "Error: 'sage -i $OPT <package>' is no longer supported, use 'sage --info <package>' instead."
    360360                exit 2;;
    361361            -f) FORCE_INSTALL=yes;;
     362            # Setting SAGE_CHECK here duplicates what we do in sage-spkg
     363            # but we need it in "make" already when there are (order-only)
     364            # dependencies on packages providing test infrastructure
     365            -c) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=yes;;
     366            -w) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=warn;;
    362367            -*) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT";;
    363368            *) PACKAGES="$PACKAGES $OPT";;
    364369        esac

comment:21 Changed 14 months ago by mkoeppe

Forgot export SAGE_CHECK obviously

comment:22 Changed 14 months ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

comment:23 Changed 14 months ago by vdelecroix

  • Status changed from needs_review to needs_work

The doctester does not like dot

Traceback (most recent call last):
  File "/opt/sage/src/bin/sage-runtests", line 177, in <module>
    DC = DocTestController(options, args)
  File "/opt/sage/local/lib/python3.7/site-packages/sage/doctest/control.py", line 365, in __init__
    raise ValueError('invalid optional tag {!r}'.format(o))
ValueError: invalid optional tag 'symengine.py'

comment:24 Changed 13 months ago by vdelecroix

  • Branch changed from u/isuruf/29497 to public/29497
  • Commit changed from 42d49e164f7ee43678ce3ec393bd37d429f9b29a to 1e57389528ff32491b637daef08cf2915b7e5c2c
  • Status changed from needs_work to needs_review

New commits:

f8154a529497: package symengine
c5b7d97distros/conda.txt file
cb0b173symengine.py also depends on cmake
4fa6c1fupstream_url
34593abchange names symengine.py -> symengine_py
9dbca0bMake sure symengine finds dependencies from SAGE_LOCAL
1e57389Fix testing

comment:25 Changed 13 months ago by git

  • Commit changed from 1e57389528ff32491b637daef08cf2915b7e5c2c to 6ef49e056b3f52587cfbf0fa38b252b01e09d42a

Branch pushed to git repo; I updated commit sha1. New commits:

6ef49e0a simple example

comment:26 Changed 13 months ago by vdelecroix

@isuruf: I think this ticket is ready for inclusion. Next step is I believe to implement the conversion Sage -> symengine (which is not there, right?)

comment:27 Changed 13 months ago by vdelecroix

  • Keywords thursdaysbdx added

comment:28 Changed 13 months ago by mkoeppe

  • Description modified (diff)

comment:29 Changed 13 months ago by mkoeppe

Does sympy have to be configured specifically to use this package? Does the package have to be available at installation time of sympy?

comment:30 Changed 13 months ago by isuruf

symengine and sympy can interoperate without any configuration.

There's a feature in sympy enabled by the env variable USE_SYMENGINE=1 at runtime which will make some modules of sympy use symengine as the core symbolic engine. This is still experimental work.

comment:31 follow-up: Changed 13 months ago by certik

@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine.

Last edited 13 months ago by certik (previous) (diff)

comment:32 Changed 13 months ago by mkoeppe

  • Status changed from needs_review to needs_work
[symengine_py-0.6.1] Installing symengine_py-0.6.1
[symengine_py-0.6.1] /Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/build/bin/sage-pip-install requires . as final argument

comment:33 Changed 13 months ago by git

  • Commit changed from 6ef49e056b3f52587cfbf0fa38b252b01e09d42a to bb7c6c2d0686930bd02100c746ce808250ba8ab2

Branch pushed to git repo; I updated commit sha1. New commits:

bb7c6c2Merge 9.2.beta0

comment:34 Changed 13 months ago by git

  • Commit changed from bb7c6c2d0686930bd02100c746ce808250ba8ab2 to 30fce9ca0d71ec31621f82fc77eb5c380b4bcba0

Branch pushed to git repo; I updated commit sha1. New commits:

30fce9cmove . at the end in sdh_pip_install

comment:35 Changed 13 months ago by vdelecroix

  • Status changed from needs_work to needs_review

comment:36 Changed 13 months ago by git

  • Commit changed from 30fce9ca0d71ec31621f82fc77eb5c380b4bcba0 to 2d0a81fe236ff03f3d90b8bcbf8398adaecc8865

Branch pushed to git repo; I updated commit sha1. New commits:

2d0a81f.github/workflows: Add symengine_py

comment:38 Changed 13 months ago by mkoeppe

  • Keywords sd109 added

comment:39 Changed 13 months ago by mkoeppe

  • Status changed from needs_review to needs_work

symengine_py build fails:

    running build_ext
    error: error in command line: command 'BuildExtWithCmake' has no such option 'no_user_cfg'
    Running setup.py install for symengine: finished with status 'error'

(see https://github.com/mkoeppe/sage/runs/728445828?check_suite_focus=true)

comment:40 Changed 11 months ago by git

  • Commit changed from 2d0a81fe236ff03f3d90b8bcbf8398adaecc8865 to 5040a042aedae753af6b35f2b88ef323393f85d0

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

178e0c529497: package symengine
3da3b6fdistros/conda.txt file
f819ee9symengine.py also depends on cmake
aa83ce3upstream_url
950f88dchange names symengine.py -> symengine_py
4b3bba8Make sure symengine finds dependencies from SAGE_LOCAL
45b60c7Fix testing
898ed4fa simple example
a331b70move . at the end in sdh_pip_install
5040a04.github/workflows: Add symengine_py

comment:41 Changed 11 months ago by mkoeppe

Rebased on 9.2.beta3

comment:42 Changed 11 months ago by git

  • Commit changed from 5040a042aedae753af6b35f2b88ef323393f85d0 to 3bc107bde4dc8d0ee3839b0eeb23a5e4c672b373

Branch pushed to git repo; I updated commit sha1. New commits:

3bc107bbuild/pkgs/symengine_py/spkg-install.in: Remove bad args to pip install

comment:44 Changed 11 months ago by mkoeppe

  • Authors changed from Vincent Delecroix, Isuru Fernando to Vincent Delecroix, Isuru Fernando, Matthias Koeppe
  • Status changed from needs_work to needs_review

comment:45 Changed 11 months ago by vdelecroix

Maybe

-upstream_url=https://github.com/symengine/symengine/releases/download/v0.6.0/symengine-VERSION.tar.gz
+upstream_url=https://github.com/symengine/symengine/releases/download/vVERSION/symengine-VERSION.tar.gz

comment:46 Changed 11 months ago by git

  • Commit changed from 3bc107bde4dc8d0ee3839b0eeb23a5e4c672b373 to 6da2383db12ed3fd2b49ab014c2737865b3c1de8

Branch pushed to git repo; I updated commit sha1. New commits:

6da2383build/pkgs/symengine/checksums.ini: More version templating

comment:47 in reply to: ↑ 43 Changed 11 months ago by mkoeppe

Replying to mkoeppe:

Tests run at https://github.com/mkoeppe/sage/actions/runs/157858355

The docker-based tests didn't go through because they are broken in 9.2.beta3 (#30064 has the fix).

But the cygwin-standard build (https://github.com/mkoeppe/sage/runs/838016220) reveals a build problem:

[arb-2.16.0.p0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/arb-2.16.0.p0.log
[ecm-7.0.4.p1] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/ecm-7.0.4.p1.log
  [ecm-7.0.4.p1] successfully installed.
  [arb-2.16.0.p0] successfully installed.
sage-logger -p 'sage-spkg -y -o  symengine-0.6.0' '/cygdrive/d/a/sage/sage/logs/pkgs/symengine-0.6.0.log'
[symengine-0.6.0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/symengine-0.6.0.log
  [symengine-0.6.0] error installing, exit status 1. End of log file:
  [symengine-0.6.0]   -- Check size of long double
  [symengine-0.6.0]   -- Check size of long double - done
  [symengine-0.6.0]   -- Found GMP: /usr/lib/libgmp.dll.a  
  [symengine-0.6.0]   -- Found ECM: /cygdrive/d/a/sage/sage/local/lib/libecm.dll.a  
  [symengine-0.6.0]   -- Found FLINT: /usr/lib/libflint.dll.a  
  [symengine-0.6.0]   CMake Error at /usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  [symengine-0.6.0]     Could NOT find ARB (missing: ARB_LIBRARIES)
  [symengine-0.6.0]   Call Stack (most recent call first):
  [symengine-0.6.0]     /usr/share/cmake-3.14.5/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  [symengine-0.6.0]     cmake/FindARB.cmake:11 (find_package_handle_standard_args)
  [symengine-0.6.0]     CMakeLists.txt:336 (find_package)
  [symengine-0.6.0]   
  [symengine-0.6.0]   
  [symengine-0.6.0]   -- Configuring incomplete, errors occurred!

comment:48 Changed 11 months ago by mkoeppe

  • Status changed from needs_review to needs_work

comment:49 follow-up: Changed 11 months ago by isuruf

This is an issue with arb not installing the import library on windows. https://github.com/fredrik-johansson/arb/pull/315 should fix it

comment:50 Changed 11 months ago by git

  • Commit changed from 6da2383db12ed3fd2b49ab014c2737865b3c1de8 to 055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b

Branch pushed to git repo; I updated commit sha1. New commits:

539c182build/make/install: Do not depend on src/bin/sage-version.sh
761092cMerge branch 't/29987/build_make_install__do_not_depend_on_src_bin_sage_version_sh' into t/30064/fix_tox_docker_builds_broken_by__29884
f2efa6asrc/doc/bootstrap: Create the directory src/doc/en/reference/repl if it does not exist
b7bf43bbuild/bin/write-dockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884
365ce61Merge branch 'u/mkoeppe/fix_tox_docker_builds_broken_by__29884' of git://trac.sagemath.org/sage into HEAD
1e7becctox.ini [debian-buster, -sid]: IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7-dev
fb61a31Merge branch 'u/mkoeppe/tox_ini__debian_bullseye___sid_have_python3_8_instead_of_3_7' of git://trac.sagemath.org/sage into 9.2.beta3+ci-fixes
055b6b4Merge branch '9.2.beta3+ci-fixes' into t/29497/public/29497

comment:51 in reply to: ↑ 49 Changed 11 months ago by mkoeppe

Replying to isuruf:

This is an issue with arb not installing the import library on windows. https://github.com/fredrik-johansson/arb/pull/315 should fix it

It's marked as a draft - should we be testing it here on the ticket?

comment:52 Changed 11 months ago by git

  • Commit changed from 055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b to 53f960de9f0e8f83c3c89033ed5fef7d8ff28c42

Branch pushed to git repo; I updated commit sha1. New commits:

53f960dMerge tag '9.2.beta4' into t/29497/public/29497

comment:53 Changed 11 months ago by git

  • Commit changed from 53f960de9f0e8f83c3c89033ed5fef7d8ff28c42 to 79b0f3af36ec4269c367d5b0acb9afd9bb602949

Branch pushed to git repo; I updated commit sha1. New commits:

bfb2fb4tox.ini: Add environment local-homebrew-usrlocal
aded967Merge commit '8195cb821e01e0cb627240d7135d5634c265cab0' of git://trac.sagemath.org/sage into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
07d657dMerge tag '9.2.beta4' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda
bd15fb9build/bin/write-dockerfile.sh, tox.ini: Do not run testsuites that need pip (ssl)
79b0f3aMerge branch 't/29929/tox_ini__add_a_macos_environment_without_homebrew__conda' into t/29497/public/29497

comment:54 Changed 11 months ago by mkoeppe

  • Dependencies set to #29929

comment:55 Changed 11 months ago by mkoeppe

  • Dependencies changed from #29929 to #29929, #30118

comment:56 Changed 11 months ago by git

  • Commit changed from 79b0f3af36ec4269c367d5b0acb9afd9bb602949 to b63ac91d9607ead738829db1065f527b84f9128e

Branch pushed to git repo; I updated commit sha1. New commits:

25393b0Handle SAGE_CHECK_PACKAGES in build/make/Makefile.in, not sage-spkg
e4ec94bMerge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29497/public/29497
b63ac91build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:57 Changed 11 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:58 in reply to: ↑ 31 Changed 5 months ago by dimpase

Replying to certik:

@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine.

How about Documentation? :-)

comment:59 Changed 5 months ago by gh-bollu

@dimpase: What kind of documentation would help? I'm also interested in pushing this along!

comment:60 Changed 5 months ago by git

  • Commit changed from b63ac91d9607ead738829db1065f527b84f9128e to 2a1566da3637703f280d4dc2ab35fcf53abbc977

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

ed8a041symengine.py also depends on cmake
b1aade0upstream_url
08b96d2change names symengine.py -> symengine_py
595983eMake sure symengine finds dependencies from SAGE_LOCAL
c60012fFix testing
2f863eaa simple example
7071c14move . at the end in sdh_pip_install
e157f84build/pkgs/symengine_py/spkg-install.in: Remove bad args to pip install
ec1d25cbuild/pkgs/symengine/checksums.ini: More version templating
2a1566dbuild/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:61 Changed 5 months ago by mkoeppe

Rebased on current develop

comment:62 Changed 5 months ago by mkoeppe

  • Dependencies #29929, #30118 deleted

comment:63 Changed 5 months ago by mkoeppe

  • Description modified (diff)

comment:64 Changed 5 months ago by git

  • Commit changed from 2a1566da3637703f280d4dc2ab35fcf53abbc977 to d614ee2dec3914a75e14fa79d7881da537faccb5

Branch pushed to git repo; I updated commit sha1. New commits:

983bcc7build/pkgs/symengine*: Change SPKG.txt to SPKG.rst
4d26ab4build/pkgs/symengine_py/install-requires.txt: New
cba3613build/pkgs/symengine_py/dependencies: Update
985e7b1build/pkgs/symengine/distros: Add more
d614ee2build/pkgs/symengine_py/distros/repology.txt: New

comment:65 Changed 5 months ago by mkoeppe

  • Reviewers changed from Matthias Koeppe to Matthias Koeppe, ...
  • Status changed from needs_work to needs_review

I've updated the package metadata. Ready for review.

comment:66 Changed 5 months ago by egourgoulhon

  • Cc egourgoulhon added

comment:67 Changed 5 months ago by mkoeppe

  • Description modified (diff)

comment:68 Changed 4 months ago by vdelecroix

  • Reviewers changed from Matthias Koeppe, ... to Matthias Koeppe, Vincent Delecroix
  • Status changed from needs_review to positive_review

Install worked fine and tests pass! Time to move forward.

comment:69 Changed 4 months ago by mkoeppe

Great, thanks.

comment:70 Changed 4 months ago by vbraun

  • Branch changed from public/29497 to d614ee2dec3914a75e14fa79d7881da537faccb5
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.