Opened 3 years ago
Closed 21 months ago
#29497 closed enhancement (fixed)
package symengine and its Python interface
Reported by:  Vincent Delecroix  Owned by:  

Priority:  major  Milestone:  sage9.3 
Component:  symbolics  Keywords:  thursdaysbdx, sd109 
Cc:  Isuru Fernando, Matthias Köppe, Eric Gourgoulhon  Merged in:  
Authors:  Vincent Delecroix, Isuru Fernando, Matthias Koeppe  Reviewers:  Matthias Koeppe, Vincent Delecroix 
Report Upstream:  N/A  Work issues:  
Branch:  d614ee2 (Commits, GitHub, GitLab)  Commit:  d614ee2dec3914a75e14fa79d7881da537faccb5 
Dependencies:  Stopgaps: 
Description (last modified by )
Symengine is a C++ symbolic engine
 that can be used as a backend in sympy (and makes it much faster)
 that implements "fast callable" for floats, GMP types, flint types, arb types, ... and accepts numpy arrays as arguments (the function is applied entrywise)
It is very strong candidate for replacing some of the symbolic we have in Sage, in particular all the fast callable.
See also:
 https://groups.google.com/d/msg/sagedevel/m8XXaoIpvIw/Eeju5I61CQAJ (discussion August 2015)
 https://groups.google.com/d/msg/sympy/bPh2Hx1HRJ4/CLyuv2c1AgAJ (discussion May 2020)
 https://groups.google.com/g/sagedevel/c/sY3zhpq8T4/m/Y2Bp0_LxAAAJ (discussion January 2021)
Change History (70)
comment:1 Changed 3 years ago by
comment:2 Changed 3 years ago by
Branch:  → public/29497 

Commit:  → 69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060 
New commits:
69e3c02  29497: package symengine

comment:3 Changed 3 years ago by
Commit:  69e3c029a3eb60f7dc8b4559dcdb2b4811ddb060 → b5304c1531c5c8ede1cd32d792951f6eedf24393 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
b5304c1  29497: package symengine

comment:4 Changed 3 years ago by
Cc:  Matthias Köppe added 

Description:  modified (diff) 
Milestone:  sage9.2 → sage9.1 
Status:  new → needs_review 
comment:6 Changed 3 years ago by
I intentionally did write upstream_url because the tarballs are from github with their tags as names.
comment:7 Changed 3 years ago by
Commit:  b5304c1531c5c8ede1cd32d792951f6eedf24393 → a008f333c08881f762067c436c046bc361ad9fa0 

Branch pushed to git repo; I updated commit sha1. New commits:
a008f33  distros/conda.txt file

comment:8 followup: 12 Changed 3 years ago by
(I wanted instead to use the tarball from PyPI for symengine.py but its name is symengine0.6.1.tar.gz
which collides with symengine tarballs)
comment:10 Changed 3 years ago by
Commit:  a008f333c08881f762067c436c046bc361ad9fa0 → 1a0b08631acd3c22a1dac32d4522f143d8bb4970 

Branch pushed to git repo; I updated commit sha1. New commits:
1a0b086  symengine.py also depends on cmake

comment:11 Changed 3 years ago by
Description:  modified (diff) 

comment:12 followup: 13 Changed 3 years ago by
Replying to vdelecroix:
(I wanted instead to use the tarball from PyPI for symengine.py but its name is
symengine0.6.1.tar.gz
which collides with symengine tarballs)
The upstream_url
code handles renaming just fine

build/pkgs/symengine.py/checksums.ini
diff git a/build/pkgs/symengine.py/checksums.ini b/build/pkgs/symengine.py/checksums.ini index ea4c44b468..489d040bcc 100644
a b tarball=symengine.pyVERSION.tar.gz 2 2 sha1=40df2b8f406b6ac4a86c83a82d31593b5738a470 3 3 md5=52b035da7851414d74f3bde83b6c2976 4 4 cksum=2829876300 5 upstream_url=https://github.com/symengine/symengine.py/archive/vVERSION.tar.gz 
build/pkgs/symengine/checksums.ini
diff git a/build/pkgs/symengine/checksums.ini b/build/pkgs/symengine/checksums.ini index 8143c8eae5..b2651f694c 100644
a b tarball=symengineVERSION.tar.gz 2 2 sha1=abd7d39b1b724f47bcdc5b1d811cf33f2c413aea 3 3 md5=3c0df2b14310467c6d45bc26a557324b 4 4 cksum=100973134 5 upstream_url=https://github.com/symengine/symengine/archive/vVERSION.tar.gz
comment:13 Changed 3 years ago by
Replying to mkoeppe:
Replying to vdelecroix:
(I wanted instead to use the tarball from PyPI for symengine.py but its name is
symengine0.6.1.tar.gz
which collides with symengine tarballs)The
upstream_url
code handles renaming just fine
Wunderbar!
comment:14 Changed 3 years ago by
Commit:  1a0b08631acd3c22a1dac32d4522f143d8bb4970 → fdf87a00b447ec768de7b8b6db1709588701bd3c 

Branch pushed to git repo; I updated commit sha1. New commits:
fdf87a0  upstream_url

comment:15 Changed 3 years ago by
Reviewers:  → Matthias Koeppe 

Status:  needs_review → positive_review 
Builds and tests OK on macOS with python3. Could you add a little bit to the ticket description please  what is it intended to be used for in Sage
comment:16 Changed 3 years ago by
Authors:  Vincent Delecroix → Vincent Delecroix, Isuru Fernando 

Branch:  public/29497 → u/isuruf/29497 
Commit:  fdf87a00b447ec768de7b8b6db1709588701bd3c → 42d49e164f7ee43678ce3ec393bd37d429f9b29a 
Status:  positive_review → needs_review 
comment:17 Changed 3 years ago by
'import symengine; symengine.test()'
uses pytest. Switched to nose
comment:18 Changed 3 years ago by
Description:  modified (diff) 

comment:19 Changed 3 years ago by
As tests are not run by default, I don't like this so much
symengine $(PYTHON)  cmake pip setuptools +symengine $(PYTHON)  cmake pip setuptools nose
@Matthias: how do we solve that when tests depend on additional libraries? (same thing actually hapenning with numpy, rpy2, etc)
comment:20 Changed 3 years ago by
Something like this (untested)?

build/pkgs/symengine.py/dependencies
a b 1 symengine $(PYTHON)  cmake pip setuptools 1 symengine $(PYTHON)  cmake pip setuptools $(and $(filterout no,$(SAGE_CHECK)), nose) 2 2 3 3  4 4 All lines of this file are ignored except the first. 
src/bin/sage
diff git a/src/bin/sage b/src/bin/sage index 10acddcd96..2106102952 100755
a b if [ "$1" = 'i' ]; then 359 359 echo >&2 "Error: 'sage i $OPT <package>' is no longer supported, use 'sage info <package>' instead." 360 360 exit 2;; 361 361 f) FORCE_INSTALL=yes;; 362 # Setting SAGE_CHECK here duplicates what we do in sagespkg 363 # but we need it in "make" already when there are (orderonly) 364 # dependencies on packages providing test infrastructure 365 c) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=yes;; 366 w) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT"; SAGE_CHECK=warn;; 362 367 *) INSTALL_OPTIONS="$INSTALL_OPTIONS $OPT";; 363 368 *) PACKAGES="$PACKAGES $OPT";; 364 369 esac
comment:22 Changed 3 years ago by
Milestone:  sage9.1 → sage9.2 

comment:23 Changed 3 years ago by
Status:  needs_review → needs_work 

The doctester does not like dot
Traceback (most recent call last): File "/opt/sage/src/bin/sageruntests", line 177, in <module> DC = DocTestController(options, args) File "/opt/sage/local/lib/python3.7/sitepackages/sage/doctest/control.py", line 365, in __init__ raise ValueError('invalid optional tag {!r}'.format(o)) ValueError: invalid optional tag 'symengine.py'
comment:24 Changed 3 years ago by
Branch:  u/isuruf/29497 → public/29497 

Commit:  42d49e164f7ee43678ce3ec393bd37d429f9b29a → 1e57389528ff32491b637daef08cf2915b7e5c2c 
Status:  needs_work → needs_review 
comment:25 Changed 3 years ago by
Commit:  1e57389528ff32491b637daef08cf2915b7e5c2c → 6ef49e056b3f52587cfbf0fa38b252b01e09d42a 

Branch pushed to git repo; I updated commit sha1. New commits:
6ef49e0  a simple example

comment:26 Changed 3 years ago by
@isuruf: I think this ticket is ready for inclusion. Next step is I believe to implement the conversion Sage > symengine
(which is not there, right?)
comment:27 Changed 3 years ago by
Keywords:  thursdaysbdx added 

comment:28 Changed 3 years ago by
Description:  modified (diff) 

comment:29 Changed 3 years ago by
Does sympy have to be configured specifically to use this package? Does the package have to be available at installation time of sympy?
comment:30 Changed 3 years ago by
symengine and sympy can interoperate without any configuration.
There's a feature in sympy enabled by the env variable USE_SYMENGINE=1
at runtime which will make some modules of sympy use symengine as the core symbolic engine. This is still experimental work.
comment:31 followup: 58 Changed 3 years ago by
@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in SymEngine
.
comment:32 Changed 3 years ago by
Status:  needs_review → needs_work 

[symengine_py0.6.1] Installing symengine_py0.6.1 [symengine_py0.6.1] /Users/mkoeppe/s/sage/sagerebasing/worktreealgebraic2018spring/build/bin/sagepipinstall requires . as final argument
comment:33 Changed 3 years ago by
Commit:  6ef49e056b3f52587cfbf0fa38b252b01e09d42a → bb7c6c2d0686930bd02100c746ce808250ba8ab2 

Branch pushed to git repo; I updated commit sha1. New commits:
bb7c6c2  Merge 9.2.beta0

comment:34 Changed 3 years ago by
Commit:  bb7c6c2d0686930bd02100c746ce808250ba8ab2 → 30fce9ca0d71ec31621f82fc77eb5c380b4bcba0 

Branch pushed to git repo; I updated commit sha1. New commits:
30fce9c  move . at the end in sdh_pip_install

comment:35 Changed 3 years ago by
Status:  needs_work → needs_review 

comment:36 Changed 3 years ago by
Commit:  30fce9ca0d71ec31621f82fc77eb5c380b4bcba0 → 2d0a81fe236ff03f3d90b8bcbf8398adaecc8865 

Branch pushed to git repo; I updated commit sha1. New commits:
2d0a81f  .github/workflows: Add symengine_py

comment:37 Changed 3 years ago by
Tests running at https://github.com/mkoeppe/sage/actions/runs/121775190
comment:38 Changed 3 years ago by
Keywords:  sd109 added 

comment:39 Changed 3 years ago by
Status:  needs_review → needs_work 

symengine_py
build fails:
running build_ext error: error in command line: command 'BuildExtWithCmake' has no such option 'no_user_cfg' Running setup.py install for symengine: finished with status 'error'
(see https://github.com/mkoeppe/sage/runs/728445828?check_suite_focus=true)
comment:40 Changed 2 years ago by
Commit:  2d0a81fe236ff03f3d90b8bcbf8398adaecc8865 → 5040a042aedae753af6b35f2b88ef323393f85d0 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
178e0c5  29497: package symengine

3da3b6f  distros/conda.txt file

f819ee9  symengine.py also depends on cmake

aa83ce3  upstream_url

950f88d  change names symengine.py > symengine_py

4b3bba8  Make sure symengine finds dependencies from SAGE_LOCAL

45b60c7  Fix testing

898ed4f  a simple example

a331b70  move . at the end in sdh_pip_install

5040a04  .github/workflows: Add symengine_py

comment:42 Changed 2 years ago by
Commit:  5040a042aedae753af6b35f2b88ef323393f85d0 → 3bc107bde4dc8d0ee3839b0eeb23a5e4c672b373 

Branch pushed to git repo; I updated commit sha1. New commits:
3bc107b  build/pkgs/symengine_py/spkginstall.in: Remove bad args to pip install

comment:43 followup: 47 Changed 2 years ago by
comment:44 Changed 2 years ago by
Authors:  Vincent Delecroix, Isuru Fernando → Vincent Delecroix, Isuru Fernando, Matthias Koeppe 

Status:  needs_work → needs_review 
comment:45 Changed 2 years ago by
Maybe
upstream_url=https://github.com/symengine/symengine/releases/download/v0.6.0/symengineVERSION.tar.gz +upstream_url=https://github.com/symengine/symengine/releases/download/vVERSION/symengineVERSION.tar.gz
comment:46 Changed 2 years ago by
Commit:  3bc107bde4dc8d0ee3839b0eeb23a5e4c672b373 → 6da2383db12ed3fd2b49ab014c2737865b3c1de8 

Branch pushed to git repo; I updated commit sha1. New commits:
6da2383  build/pkgs/symengine/checksums.ini: More version templating

comment:47 Changed 2 years ago by
Replying to mkoeppe:
Tests run at https://github.com/mkoeppe/sage/actions/runs/157858355
The dockerbased tests didn't go through because they are broken in 9.2.beta3 (#30064 has the fix).
But the cygwinstandard
build (https://github.com/mkoeppe/sage/runs/838016220) reveals a build problem:
[arb2.16.0.p0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/arb2.16.0.p0.log [ecm7.0.4.p1] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/ecm7.0.4.p1.log [ecm7.0.4.p1] successfully installed. [arb2.16.0.p0] successfully installed. sagelogger p 'sagespkg y o symengine0.6.0' '/cygdrive/d/a/sage/sage/logs/pkgs/symengine0.6.0.log' [symengine0.6.0] installing. Log file: /cygdrive/d/a/sage/sage/logs/pkgs/symengine0.6.0.log [symengine0.6.0] error installing, exit status 1. End of log file: [symengine0.6.0]  Check size of long double [symengine0.6.0]  Check size of long double  done [symengine0.6.0]  Found GMP: /usr/lib/libgmp.dll.a [symengine0.6.0]  Found ECM: /cygdrive/d/a/sage/sage/local/lib/libecm.dll.a [symengine0.6.0]  Found FLINT: /usr/lib/libflint.dll.a [symengine0.6.0] CMake Error at /usr/share/cmake3.14.5/Modules/FindPackageHandleStandardArgs.cmake:137 (message): [symengine0.6.0] Could NOT find ARB (missing: ARB_LIBRARIES) [symengine0.6.0] Call Stack (most recent call first): [symengine0.6.0] /usr/share/cmake3.14.5/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE) [symengine0.6.0] cmake/FindARB.cmake:11 (find_package_handle_standard_args) [symengine0.6.0] CMakeLists.txt:336 (find_package) [symengine0.6.0] [symengine0.6.0] [symengine0.6.0]  Configuring incomplete, errors occurred!
comment:48 Changed 2 years ago by
Status:  needs_review → needs_work 

comment:49 followup: 51 Changed 2 years ago by
This is an issue with arb not installing the import library on windows. https://github.com/fredrikjohansson/arb/pull/315 should fix it
comment:50 Changed 2 years ago by
Commit:  6da2383db12ed3fd2b49ab014c2737865b3c1de8 → 055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b 

Branch pushed to git repo; I updated commit sha1. New commits:
539c182  build/make/install: Do not depend on src/bin/sageversion.sh

761092c  Merge branch 't/29987/build_make_install__do_not_depend_on_src_bin_sage_version_sh' into t/30064/fix_tox_docker_builds_broken_by__29884

f2efa6a  src/doc/bootstrap: Create the directory src/doc/en/reference/repl if it does not exist

b7bf43b  build/bin/writedockerfile.sh: ADD src/bin for bootstrapping, needed by src/doc/bootstrap after #29884

365ce61  Merge branch 'u/mkoeppe/fix_tox_docker_builds_broken_by__29884' of git://trac.sagemath.org/sage into HEAD

1e7becc  tox.ini [debianbuster, sid]: IGNORE_MISSING_SYSTEM_PACKAGES=yes because of libpython3.7dev

fb61a31  Merge branch 'u/mkoeppe/tox_ini__debian_bullseye___sid_have_python3_8_instead_of_3_7' of git://trac.sagemath.org/sage into 9.2.beta3+cifixes

055b6b4  Merge branch '9.2.beta3+cifixes' into t/29497/public/29497

comment:51 Changed 2 years ago by
Replying to isuruf:
This is an issue with arb not installing the import library on windows. https://github.com/fredrikjohansson/arb/pull/315 should fix it
It's marked as a draft  should we be testing it here on the ticket?
comment:52 Changed 2 years ago by
Commit:  055b6b490c3ad49cae9cd06e609b2c0cb97b5c3b → 53f960de9f0e8f83c3c89033ed5fef7d8ff28c42 

Branch pushed to git repo; I updated commit sha1. New commits:
53f960d  Merge tag '9.2.beta4' into t/29497/public/29497

comment:53 Changed 2 years ago by
Commit:  53f960de9f0e8f83c3c89033ed5fef7d8ff28c42 → 79b0f3af36ec4269c367d5b0acb9afd9bb602949 

Branch pushed to git repo; I updated commit sha1. New commits:
bfb2fb4  tox.ini: Add environment localhomebrewusrlocal

aded967  Merge commit '8195cb821e01e0cb627240d7135d5634c265cab0' of git://trac.sagemath.org/sage into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda

07d657d  Merge tag '9.2.beta4' into t/29929/tox_ini__add_a_macos_environment_without_homebrew__conda

bd15fb9  build/bin/writedockerfile.sh, tox.ini: Do not run testsuites that need pip (ssl)

79b0f3a  Merge branch 't/29929/tox_ini__add_a_macos_environment_without_homebrew__conda' into t/29497/public/29497

comment:54 Changed 2 years ago by
Dependencies:  → #29929 

comment:55 Changed 2 years ago by
Dependencies:  #29929 → #29929, #30118 

comment:56 Changed 2 years ago by
Commit:  79b0f3af36ec4269c367d5b0acb9afd9bb602949 → b63ac91d9607ead738829db1065f527b84f9128e 

Branch pushed to git repo; I updated commit sha1. New commits:
25393b0  Handle SAGE_CHECK_PACKAGES in build/make/Makefile.in, not sagespkg

e4ec94b  Merge branch 't/30118/handle_sage_check_packages_in_build_make_makefile_in__not_sage_spkg' into t/29497/public/29497

b63ac91  build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:57 Changed 2 years ago by
Milestone:  sage9.2 → sage9.3 

comment:58 Changed 23 months ago by
Replying to certik:
@vdelecroix, thanks for this effort. Let us know what you need, we'll be happy to implement in
SymEngine
.
How about Documentation? :)
comment:59 Changed 23 months ago by
@dimpase: What kind of documentation would help? I'm also interested in pushing this along!
comment:60 Changed 23 months ago by
Commit:  b63ac91d9607ead738829db1065f527b84f9128e → 2a1566da3637703f280d4dc2ab35fcf53abbc977 

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
ed8a041  symengine.py also depends on cmake

b1aade0  upstream_url

08b96d2  change names symengine.py > symengine_py

595983e  Make sure symengine finds dependencies from SAGE_LOCAL

c60012f  Fix testing

2f863ea  a simple example

7071c14  move . at the end in sdh_pip_install

e157f84  build/pkgs/symengine_py/spkginstall.in: Remove bad args to pip install

ec1d25c  build/pkgs/symengine/checksums.ini: More version templating

2a1566d  build/pkgs/symengine_py/dependencies: Conditionalize nose on SAGE_CHECK_symengine_py

comment:62 Changed 23 months ago by
Dependencies:  #29929, #30118 

comment:63 Changed 23 months ago by
Description:  modified (diff) 

comment:64 Changed 23 months ago by
Commit:  2a1566da3637703f280d4dc2ab35fcf53abbc977 → d614ee2dec3914a75e14fa79d7881da537faccb5 

Branch pushed to git repo; I updated commit sha1. New commits:
983bcc7  build/pkgs/symengine*: Change SPKG.txt to SPKG.rst

4d26ab4  build/pkgs/symengine_py/installrequires.txt: New

cba3613  build/pkgs/symengine_py/dependencies: Update

985e7b1  build/pkgs/symengine/distros: Add more

d614ee2  build/pkgs/symengine_py/distros/repology.txt: New

comment:65 Changed 23 months ago by
Reviewers:  Matthias Koeppe → Matthias Koeppe, ... 

Status:  needs_work → needs_review 
I've updated the package metadata. Ready for review.
comment:66 Changed 23 months ago by
Cc:  Eric Gourgoulhon added 

comment:67 Changed 23 months ago by
Description:  modified (diff) 

comment:68 Changed 22 months ago by
Reviewers:  Matthias Koeppe, ... → Matthias Koeppe, Vincent Delecroix 

Status:  needs_review → positive_review 
Install worked fine and tests pass! Time to move forward.
comment:70 Changed 21 months ago by
Branch:  public/29497 → d614ee2dec3914a75e14fa79d7881da537faccb5 

Resolution:  → fixed 
Status:  positive_review → closed 
Let me know if you run into any trouble.
Here are some of the options I use in conda. https://github.com/condaforge/symenginefeedstock/blob/master/recipe/build.sh#L6L19. They should work fine for sage (except for WITH_LLVM=yes)
Dependencies: CMake, GMP/MPIR, MPFR, MPC, FLINT2, ARB