Opened 2 years ago
Closed 18 months ago
#29497 closed enhancement (fixed)
package symengine and its Python interface
Reported by:  vdelecroix  Owned by:  

Priority:  major  Milestone:  sage9.3 
Component:  symbolics  Keywords:  thursdaysbdx, sd109 
Cc:  isuruf, mkoeppe, egourgoulhon  Merged in:  
Authors:  Vincent Delecroix, Isuru Fernando, Matthias Koeppe  Reviewers:  Matthias Koeppe, Vincent Delecroix 
Report Upstream:  N/A  Work issues:  
Branch:  d614ee2 (Commits, GitHub, GitLab)  Commit:  d614ee2dec3914a75e14fa79d7881da537faccb5 
Dependencies:  Stopgaps: 
Description (last modified by )
Symengine is a C++ symbolic engine
 that can be used as a backend in sympy (and makes it much faster)
 that implements "fast callable" for floats, GMP types, flint types, arb types, ... and accepts numpy arrays as arguments (the function is applied entrywise)
It is very strong candidate for replacing some of the symbolic we have in Sage, in particular all the fast callable.
See also:
 https://groups.google.com/d/msg/sagedevel/m8XXaoIpvIw/Eeju5I61CQAJ (discussion August 2015)
 https://groups.google.com/d/msg/sympy/bPh2Hx1HRJ4/CLyuv2c1AgAJ (discussion May 2020)
 https://groups.google.com/g/sagedevel/c/sY3zhpq8T4/m/Y2Bp0_LxAAAJ (discussion January 2021)
Change History (70)
comment:1 Changed 2 years ago by
comment:2 Changed 2 years ago by
 Branch set to public/29497
 Commit set to 69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060
New commits:
69e3c02  29497: package symengine

comment:3 Changed 2 years ago by
 Commit changed from 69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060 to b5304c1531c5c8ede1cd32d792951f6eedf24393
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
b5304c1  29497: package symengine

comment:4 Changed 2 years ago by
 Cc mkoeppe added
 Description modified (diff)
 Milestone changed from sage9.2 to sage9.1
 Status changed from new to needs_review
comment:5 Changed 2 years ago by
Please add upstream_url to checksums.ini and distros/ Information
comment:6 Changed 2 years ago by
I intentionally did write upstream_url because the tarballs are from github with their tags as names.
comment:7 Changed 2 years ago by
 Commit changed from b5304c1531c5c8ede1cd32d792951f6eedf24393 to a008f333c08881f762067c436c046bc361ad9fa0
Branch pushed to git repo; I updated commit sha1. New commits:
a008f33  distros/conda.txt file

comment:8 followup: ↓ 12 Changed 2 years ago by
(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine0.6.1.tar.gz
which collides with symengine tarballs)
comment:9 Changed 2 years ago by
symengine.py
needs cmake as well.
comment:10 Changed 2 years ago by
 Commit changed from a008f333c08881f762067c436c046bc361ad9fa0 to 1a0b08631acd3c22a1dac32d4522f143d8bb4970
Branch pushed to git repo; I updated commit sha1. New commits:
1a0b086  symengine.py also depends on cmake

comment:11 Changed 2 years ago by
 Description modified (diff)
comment:12 in reply to: ↑ 8 ; followup: ↓ 13 Changed 2 years ago by
Replying to vdelecroix:
(I wanted instead to use the tarball from PyPI for symengine.py but its name is
symengine0.6.1.tar.gz
which collides with symengine tarballs)
The upstream_url
code handles renaming just fine

build/pkgs/symengine.py/checksums.ini
diff git a/build/pkgs/symengine.py/checksums.ini b/build/pkgs/symengine.py/checksums.ini index ea4c44b468..489d040bcc 100644
a b tarball=symengine.pyVERSION.tar.gz 2 2 sha1=40df2b8f406b6ac4a86c83a82d31593b5738a470 3 3 md5=52b035da7851414d74f3bde83b6c2976 4 4 cksum=2829876300 5 upstream_url=https://github.com/symengine/symengine.py/archive/vVERSION.tar.gz 
build/pkgs/symengine/checksums.ini
diff git a/build/pkgs/symengine/checksums.ini b/build/pkgs/symengine/checksums.ini index 8143c8eae5..b2651f694c 100644
a b tarball=symengineVERSION.tar.gz 2 2 sha1=abd7d39b1b724f47bcdc5b1d811cf33f2c413aea 3 3 md5=3c0df2b14310467c6d45bc26a557324b 4 4 cksum=100973134 5 upstream_url=https://github.com/symengine/symengine/archive/vVERSION.tar.gz
comment:13 in reply to: ↑ 12 Changed 2 years ago by
Replying to mkoeppe:
Replying to vdelecroix:
(I wanted instead to use the tarball from PyPI for symengine.py but its name is
symengine0.6.1.tar.gz
which collides with symengine tarballs)The
upstream_url
code handles renaming just fine
Wunderbar!
comment:14 Changed 2 years ago by
 Commit changed from 1a0b08631acd3c22a1dac32d4522f143d8bb4970 to fdf87a00b447ec768de7b8b6db1709588701bd3c
Branch pushed to git repo; I updated commit sha1. New commits:
fdf87a0  upstream_url

comment:15 Changed 2 years ago by
 Reviewers set to Matthias Koeppe
 Status changed from needs_review to positive_review
Builds and tests OK on macOS with python3. Could you add a little bit to the ticket description please  what is it intended to be used for in Sage
comment:16 Changed 2 years ago by
 Branch changed from public/29497 to u/isuruf/29497
 Commit changed from fdf87a00b447ec768de7b8b6db1709588701bd3c to 42d49e164f7ee43678ce3ec393bd37d429f9b29a
 Status changed from positive_review to needs_review
comment:17 Changed 2 years ago by
'import symengine; symengine.test()'
uses pytest. Switched to nose
comment:18 Changed 2 years ago by
 Description modified (diff)
comment:19 Changed 2 years ago by
As tests are not run by default, I don't like this so much
symengine $(PYTHON)  cmake pip setuptools +symengine $(PYTHON)  cmake pip setuptools nose
@Matthias: how do we solve that when tests depend on additional libraries? (same thing actually hapenning with numpy, rpy2, etc)
comment:20 Changed 2 years ago by
Something like this (untested)?

build/pkgs/symengine.py/dependencies
a b 1 symengine $(PYTHON)  cmake pip setuptools 1 symengine $(PYTHON)  cmake pip setuptools $(and $(filterout no,$(SAGE_CHECK)), nose) 2 2 3 3  4 4 All lines of this file are ignored except the first. 
src/bin/sage
diff git a/src/bin/sage b/src/bin/sage index 10acddcd96..2106102952 100755
a b if [ "$1" = 'i' ]; then 359 359 echo >&2 "Error: 'sage i $OPT <package>' is no longer supported, use 'sage info <package>' instead." 360 360 exit 2;; 361 361 f) FORCE_INSTALL=yes;; 362 # Setting SAGE_CHECK here duplicates what we do in sagespkg 363 # but we need it in "make" already when there are (orderonly) 364 # dependencies on packages providing test infrastructure 365 c) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=yes;; 366 w) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=warn;; 362 367 *) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT";; 363 368 *) PACKAGES="$PACKAGES $OPT";; 364 369 esac
comment:21 Changed 2 years ago by
Forgot export SAGE_CHECK
obviously
comment:22 Changed 2 years ago by
 Milestone changed from sage9.1 to sage9.2
comment:23 Changed 2 years ago by
 Status changed from needs_review to needs_work
The doctester does not like dot
Traceback (most recent call last): File "/opt/sage/src/bin/sageruntests", line 177, in <module> DC = DocTestController(options, args) File "/opt/sage/local/lib/python3.7/sitepackages/sage/doctest/control.py", line 365, in __init__ raise ValueError('invalid optional tag {!r}'.format(o)) ValueError: invalid optional tag 'symengine.py'
comment:24 Changed 2 years ago by
 Branch changed from u/isuruf/29497 to public/29497
 Commit changed from 42d49e164f7ee43678ce3ec393bd37d429f9b29a to 1e57389528ff32491b637daef08cf2915b7e5c2c
 Status changed from needs_work to needs_review
comment:25 Changed 2 years ago by
 Commit changed from 1e57389528ff32491b637daef08cf2915b7e5c2c to 6ef49e056b3f52587cfbf0fa38b252b01e09d42a
Branch pushed to git repo; I updated commit sha1. New commits:
6ef49e0  a simple example

comment:26 Changed 2 years ago by
@isuruf: I think this ticket is ready for inclusion. Next step is I believe to implement the conversion Sage > symengine
(which is not there, right?)
comment:27 Changed 2 years ago by
 Keywords thursdaysbdx added
comment:28 Changed 2 years ago by
 Description modified (diff)
comment:29 Changed 2 years ago by
Does sympy have to be configured specifically to use this package? Does the package have to be available at installation time of sympy?
comment:30 Changed 2 years ago by
symengine and sympy can interoperate without any configuration.
There's a feature in sympy enabled by the env variable USE_SYMENGINE=1
at runtime which will make some modules of sympy use symengine as the core symbolic engine. This is still experimental work.
comment:31 followup: ↓ 58 Changed 2 years ago by
@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine?.
comment:32 Changed 2 years ago by
 Status changed from needs_review to needs_work
[symengine_py0.6.1] Installing symengine_py0.6.1 [symengine_py0.6.1] /Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/build/bin/sagepipinstall requires . as final argument
comment:33 Changed 2 years ago by
 Commit changed from 6ef49e056b3f52587cfbf0fa38b252b01e09d42a to bb7c6c2d0686930bd02100c746ce808250ba8ab2
Branch pushed to git repo; I updated commit sha1. New commits:
bb7c6c2  Merge 9.2.beta0

comment:34 Changed 2 years ago by
 Commit changed from bb7c6c2d0686930bd02100c746ce808250ba8ab2 to 30fce9ca0d71ec31621f82fc77eb5c380b4bcba0
Branch pushed to git repo; I updated commit sha1. New commits:
30fce9c  move . at the end in sdh_pip_install

comment:35 Changed 2 years ago by
 Status changed from needs_work to needs_review
comment:36 Changed 2 years ago by
 Commit changed from 30fce9ca0d71ec31621f82fc77eb5c380b4bcba0 to 2d0a81fe236ff03f3d90b8bcbf8398adaecc8865
Branch pushed to git repo; I updated commit sha1. New commits:
2d0a81f  .github/workflows: Add symengine_py

comment:37 Changed 2 years ago by
Tests running at https://github.com/mkoeppe/sage/actions/runs/121775190
comment:38 Changed 2 years ago by
 Keywords sd109 added
comment:39 Changed 2 years ago by
 Status changed from needs_review to needs_work
symengine_py
build fails:
running build_ext error: error in command line: command 'BuildExtWithCmake' has no such option 'no_user_cfg' Running setup.py install for symengine: finished with status 'error'
(see https://github.com/mkoeppe/sage/runs/728445828?check_suite_focus=true)
comment:40 Changed 2 years ago by
 Commit changed from 2d0a81fe236ff03f3d90b8bcbf8398adaecc8865 to 5040a042aedae753af6b35f2b88ef323393f85d0
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
178e0c5  29497: package symengine

3da3b6f  distros/conda.txt file

f819ee9  symengine.py also depends on cmake

aa83ce3  upstream_url

950f88d  change names symengine.py > symengine_py

4b3bba8  Make sure symengine finds dependencies from SAGE_LOCAL

45b60c7  Fix testing

898ed4f  a simple example

a331b70  move . at the end in sdh_pip_install

5040a04  .github/workflows: Add symengine_py

comment:41 Changed 2 years ago by
Rebased on 9.2.beta3
comment:42 Changed 2 years ago by
 Commit changed from 5040a042aedae753af6b35f2b88ef323393f85d0 to 3bc107bde4dc8d0ee3839b0eeb23a5e4c672b373
Branch pushed to git repo; I updated commit sha1. New commits:
3bc107b  build/pkgs/symengine_py/spkginstall.in: Remove bad args to pip install

comment:43 followup: ↓ 47 Changed 2 years ago by
comment:44 Changed 2 years ago by
 Status changed from needs_work to needs_review
comment:45 Changed 2 years ago by
Maybe
upstream_url=https://github.com/symengine/symengine/releases/download/v0.6.0/symengineVERSION.tar.gz +upstream_url=https://github.com/symengine/symengine/releases/download/vVERSION/symengineVERSION.tar.gz
comment:46 Changed 2 years ago by
 Commit changed from 3bc107bde4dc8d0ee3839b0eeb23a5e4c672b373 to 6da2383db12ed3fd2b49ab014c2737865b3c1de8
Branch pushed to git repo; I updated commit sha1. New commits:
6da2383  build/pkgs/symengine/checksums.ini: More version templating

comment:47 in reply to: ↑ 43 Changed 2 years ago by
Replying to mkoeppe:
Tests run at https://github.com/mkoeppe/sage/actions/runs/157858355
The dockerbased tests didn't go through because they are broken in 9.2.beta3 (#30064 has the fix).
But the cygwinstandard
build (https://github.com/mkoeppe/sage/runs/838016220) reveals a build problem:
[arb2.16.0.p0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/arb2.16.0.p0.log [ecm7.0.4.p1] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/ecm7.0.4.p1.log [ecm7.0.4.p1] successfully installed. [arb2.16.0.p0] successfully installed. sagelogger p 'sagespkg y o symengine0.6.0' '/cygdrive/d/a/sage/sage/logs/pkgs/symengine0.6.0.log' [symengine0.6.0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/symengine0.6.0.log [symengine0.6.0] error installing, exit status 1. End of log file: [symengine0.6.0]  Check size of long double [symengine0.6.0]  Check size of long double  done [symengine0.6.0]  Found GMP: /usr/lib/libgmp.dll.a [symengine0.6.0]  Found ECM: /cygdrive/d/a/sage/sage/local/lib/libecm.dll.a [symengine0.6.0]  Found FLINT: /usr/lib/libflint.dll.a [symengine0.6.0] CMake Error at /usr/share/cmake3.14.5/Modules/FindPackageHandleStandardArgs.cmake:137 (message): [symengine0.6.0] Could NOT find ARB (missing: ARB_LIBRARIES) [symengine0.6.0] Call Stack (most recent call first): [symengine0.6.0] /usr/share/cmake3.14.5/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE) [symengine0.6.0] cmake/FindARB.cmake:11 (find_package_handle_standard_args) [symengine0.6.0] CMakeLists.txt:336 (find_package) [symengine0.6.0] [symengine0.6.0] [symengine0.6.0]  Configuring incomplete, errors occurred!
comment:48 Changed 2 years ago by
 Status changed from needs_review to needs_work
comment:49 followup: ↓ 51 Changed 2 years ago by
This is an issue with arb not installing the import library on windows. https://github.com/fredrikjohansson/arb/pull/315 should fix it
comment:50 Changed 2 years ago by
 Commit changed from 6da2383db12ed3fd2b49ab014c2737865b3c1de8 to 055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b
Branch pushed to git repo; I updated commit sha1. New commits:
539c182  build/make/install: Do not depend on src/bin/sageversion.sh

761092c  Merge branch 't/29987/build_make_install__do_not_depend_on_src_bin_sage_version_sh' into t/30064/fix_tox_docker_builds_broken_by__29884

f2efa6a  src/doc/bootstrap: Create the directory src/doc/en/reference/repl if it does not exist

b7bf43b  build/bin/writedockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884

365ce61  Merge branch 'u/mkoeppe/fix_tox_docker_builds_broken_by__29884' of git://trac.sagemath.org/sage into HEAD

1e7becc  tox.ini [debianbuster, sid]: IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7dev

fb61a31  Merge branch 'u/mkoeppe/tox_ini__debian_bullseye___sid_have_python3_8_instead_of_3_7' of git://trac.sagemath.org/sage into 9.2.beta3+cifixes

055b6b4  Merge branch '9.2.beta3+cifixes' into t/29497/public/29497

comment:51 in reply to: ↑ 49 Changed 2 years ago by
Replying to isuruf:
This is an issue with arb not installing the import library on windows. https://github.com/fredrikjohansson/arb/pull/315 should fix it
It's marked as a draft  should we be testing it here on the ticket?
comment:52 Changed 2 years ago by
 Commit changed from 055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b to 53f960de9f0e8f83c3c89033ed5fef7d8ff28c42
Branch pushed to git repo; I updated commit sha1. New commits:
53f960d  Merge tag '9.2.beta4' into t/29497/public/29497

comment:53 Changed 2 years ago by
 Commit changed from 53f960de9f0e8f83c3c89033ed5fef7d8ff28c42 to 79b0f3af36ec4269c367d5b0acb9afd9bb602949
Branch pushed to git repo; I updated commit sha1. New commits:
bfb2fb4  tox.ini: Add environment localhomebrewusrlocal

aded967  Merge commit '8195cb821e01e0cb627240d7135d5634c265cab0' of git://trac.sagemath.org/sage into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda

07d657d  Merge tag '9.2.beta4' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda

bd15fb9  build/bin/writedockerfile.sh, tox.ini: Do not run testsuites that need pip (ssl)

79b0f3a  Merge branch 't/29929/tox_ini__add_a_macos_environment_without_homebrew__conda' into t/29497/public/29497

comment:54 Changed 2 years ago by
 Dependencies set to #29929
comment:55 Changed 2 years ago by
 Dependencies changed from #29929 to #29929, #30118
comment:56 Changed 2 years ago by
 Commit changed from 79b0f3af36ec4269c367d5b0acb9afd9bb602949 to b63ac91d9607ead738829db1065f527b84f9128e
Branch pushed to git repo; I updated commit sha1. New commits:
25393b0  Handle SAGE_CHECK_PACKAGES in build/make/Makefile.in, not sagespkg

e4ec94b  Merge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29497/public/29497

b63ac91  build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:57 Changed 2 years ago by
 Milestone changed from sage9.2 to sage9.3
comment:58 in reply to: ↑ 31 Changed 19 months ago by
Replying to certik:
@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in
SymEngine
.
How about Documentation? :)
comment:59 Changed 19 months ago by
@dimpase: What kind of documentation would help? I'm also interested in pushing this along!
comment:60 Changed 19 months ago by
 Commit changed from b63ac91d9607ead738829db1065f527b84f9128e to 2a1566da3637703f280d4dc2ab35fcf53abbc977
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
ed8a041  symengine.py also depends on cmake

b1aade0  upstream_url

08b96d2  change names symengine.py > symengine_py

595983e  Make sure symengine finds dependencies from SAGE_LOCAL

c60012f  Fix testing

2f863ea  a simple example

7071c14  move . at the end in sdh_pip_install

e157f84  build/pkgs/symengine_py/spkginstall.in: Remove bad args to pip install

ec1d25c  build/pkgs/symengine/checksums.ini: More version templating

2a1566d  build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:61 Changed 19 months ago by
Rebased on current develop
comment:62 Changed 19 months ago by
 Dependencies #29929, #30118 deleted
comment:63 Changed 19 months ago by
 Description modified (diff)
comment:64 Changed 19 months ago by
 Commit changed from 2a1566da3637703f280d4dc2ab35fcf53abbc977 to d614ee2dec3914a75e14fa79d7881da537faccb5
Branch pushed to git repo; I updated commit sha1. New commits:
983bcc7  build/pkgs/symengine*: Change SPKG.txt to SPKG.rst

4d26ab4  build/pkgs/symengine_py/installrequires.txt: New

cba3613  build/pkgs/symengine_py/dependencies: Update

985e7b1  build/pkgs/symengine/distros: Add more

d614ee2  build/pkgs/symengine_py/distros/repology.txt: New

comment:65 Changed 19 months ago by
 Reviewers changed from Matthias Koeppe to Matthias Koeppe, ...
 Status changed from needs_work to needs_review
I've updated the package metadata. Ready for review.
comment:66 Changed 19 months ago by
 Cc egourgoulhon added
comment:67 Changed 19 months ago by
 Description modified (diff)
comment:68 Changed 18 months ago by
 Reviewers changed from Matthias Koeppe, ... to Matthias Koeppe, Vincent Delecroix
 Status changed from needs_review to positive_review
Install worked fine and tests pass! Time to move forward.
comment:69 Changed 18 months ago by
Great, thanks.
comment:70 Changed 18 months ago by
 Branch changed from public/29497 to d614ee2dec3914a75e14fa79d7881da537faccb5
 Resolution set to fixed
 Status changed from positive_review to closed
Let me know if you run into any trouble.
Here are some of the options I use in conda. https://github.com/condaforge/symenginefeedstock/blob/master/recipe/build.sh#L6L19. They should work fine for sage (except for WITH_LLVM=yes)
Dependencies: CMake, GMP/MPIR, MPFR, MPC, FLINT2, ARB