#22704 closed enhancement (fixed)
Create a variant of the polymake interface using polymake's callable library (polymake::Main) via JuPyMake
Reported by:  mkoeppe  Owned by:  

Priority:  major  Milestone:  sage8.8 
Component:  interfaces  Keywords:  days84 
Cc:  SimonKing, jipilab, ghsophiasage, ghLaisRast, ghkliem, ghsebasguts, vdelecroix, tscrim  Merged in:  
Authors:  Matthias Koeppe  Reviewers:  Dima Pasechnik 
Report Upstream:  Reported upstream. No feedback yet.  Work issues:  
Branch:  8d9e2c2 (Commits, GitHub, GitLab)  Commit:  
Dependencies:  #24905  Stopgaps: 
Description (last modified by )
Following up on #22452 (Create a Polymake pexpect interface):
As suggested by Gawrilow in https://forum.polymake.org/viewtopic.php?f=8&t=544, we could create a variant of the polymake pexpect interface that does not go through pty's but rather replaces the relevant send/receive methods of pexpect by calls to simulate_shell_input
(linking to libpolymake).
Recent polymake versions have developed this further and now provide a class polymake::Main
with convenient access to the polymake interpreter. We use the (partial) Python bindings for this, https://github.com/polymake/JuPyMake which are used for Polymake's jupyter.
Upstream package:
wget P upstream https://files.pythonhosted.org/packages/32/dd/0a48546daa2461e57207d31eeb64a3ad77fd9fb339d90e1e067ea25991a5/JuPyMake0.9.tar.gz
We do want to keep the current ptybased pexpect interface, though, because it may be handy for running polymake on a remote server; or for guarding against a crashing polymake (sigh).
Remaining issue: #27807
See also: #22710: Metaticket: polymake
Change History (59)
comment:1 Changed 4 years ago by
 Dependencies set to #22452
 Description modified (diff)
comment:2 Changed 3 years ago by
 Description modified (diff)
 Milestone changed from sage8.0 to sage8.4
 Summary changed from Create a variant of the polymake pexpect interface using polymake's simulate_shell_input to Create a variant of the polymake pexpect interface using polymake's simulate_shell_input / JuPyMake
comment:3 Changed 3 years ago by
 Dependencies changed from #22452 to #22452, #26333
comment:4 Changed 2 years ago by
 Branch set to public/22704
comment:5 Changed 2 years ago by
 Cc jipilab ghsophiasage ghLaisRast ghkliem ghsebasguts vdelecroix added
 Commit set to 2f855b0298f80080013d7c9b1a9f58b98876d1c8
 Description modified (diff)
comment:6 Changed 2 years ago by
 Commit changed from 2f855b0298f80080013d7c9b1a9f58b98876d1c8 to 78b5036ac44d171568aa15774190ed3e20cb7c17
Branch pushed to git repo; I updated commit sha1. New commits:
78b5036  More doctests etc.

comment:7 Changed 2 years ago by
 Description modified (diff)
comment:8 Changed 2 years ago by
 Description modified (diff)
comment:9 Changed 2 years ago by
 Commit changed from 78b5036ac44d171568aa15774190ed3e20cb7c17 to 92274e0fd34e47c1db95bd45a6620755362e9e43
Branch pushed to git repo; I updated commit sha1. New commits:
92274e0  Add package jupymake

comment:10 Changed 2 years ago by
Some polymake errors are not properly signalled by Polymake: Only the stderr slot [2] is filled but the error slot [3] is empty
sage: from JuPyMake import ExecuteCommand sage: ExecuteCommand('help("Triangulation");') (True, '', "polymake: ERROR: unknown help topic 'Triangulation'\n", '')
comment:11 Changed 2 years ago by
 Commit changed from 92274e0fd34e47c1db95bd45a6620755362e9e43 to 6bd4971d19129c9edbba0087f8dc452f9908ae3f
comment:12 Changed 2 years ago by
 Commit changed from 6bd4971d19129c9edbba0087f8dc452f9908ae3f to 0bad10c608fa88f230b4244470207b1ef2de61a3
Branch pushed to git repo; I updated commit sha1. New commits:
0bad10c  Fix jupymake help

comment:13 Changed 2 years ago by
 Commit changed from 0bad10c608fa88f230b4244470207b1ef2de61a3 to 67086632c142d1dc4df8517ff38c55eb5c1174ca
comment:14 Changed 2 years ago by
 Commit changed from 67086632c142d1dc4df8517ff38c55eb5c1174ca to a614876d3b8981641c7acdd08a3612ff27bc8319
Branch pushed to git repo; I updated commit sha1. New commits:
a5c8eb8  Add optional  polymake

ad2b976  Can only quit polymake in pexpect interface

7ec6c74  Mark jupymake tests optional  jupymake

8b7ffbd  Select polymake interface based on feature test

a614876  Fixup polymake doctests that depend on 'used package' messages

comment:15 Changed 2 years ago by
 Dependencies changed from #22452, #26333 to #26333
 Description modified (diff)
comment:16 Changed 2 years ago by
Signal handling issues are solved in new version of JuPyMake?  waiting for a new release
comment:17 Changed 2 years ago by
This is coming from #24905, which is already set as positive review.
Could you correct the following pyflakes warnings:
+src/sage/geometry/polyhedron/backend_polymake.py:29: 'sage.rings.all.QQ' imported but unused +src/sage/geometry/polyhedron/backend_polymake.py:29: 'sage.rings.all.ZZ' imported but unused
comment:18 Changed 2 years ago by
 Commit changed from a614876d3b8981641c7acdd08a3612ff27bc8319 to bcc7c6b88bb6c622988bf3b06579a5560c0181ee
Branch pushed to git repo; I updated commit sha1. New commits:
bcc7c6b  Add polymake patch for thread correctness

comment:20 Changed 2 years ago by
 Dependencies #26333 deleted
comment:21 Changed 2 years ago by
 Description modified (diff)
comment:22 Changed 2 years ago by
 Commit changed from bcc7c6b88bb6c622988bf3b06579a5560c0181ee to 1b8e2a580423e3b215930b98d065d4514af02af9
comment:23 Changed 2 years ago by
 Dependencies set to #24905
 Description modified (diff)
Signal handling fixed by jupymake update
comment:24 Changed 2 years ago by
 Description modified (diff)
 Report Upstream changed from N/A to Reported upstream. No feedback yet.
polymake's error handling bug is reported upstream.
comment:25 Changed 2 years ago by
 Cc tscrim added
 Status changed from new to needs_review
Apart from 1 remaining doctest regarding the form of some error messages, this is ready for review!
comment:26 Changed 2 years ago by
 Description modified (diff)
 Milestone changed from sage8.4 to sage8.8
 Summary changed from Create a variant of the polymake pexpect interface using polymake's simulate_shell_input / JuPyMake to Create a variant of the polymake interface using polymake's callable library (polymake::Main) via JuPyMake
comment:27 Changed 2 years ago by
 Commit changed from 1b8e2a580423e3b215930b98d065d4514af02af9 to e46cd380e9645b0e7e6a26474aa2f69999f7cdc3
Branch pushed to git repo; I updated commit sha1. New commits:
666db96  Added two missing optional tags

4e91dcc  Change format of a 'not tested' doc test annotation

05f3e61  update docs, as discussed on #24905

53c8ee6  Mention libtermreadkeyperl

b613e57  Mention libtermreadlinegnuperl

e5c3249  polymake: Mention MOngoDB for polydb

b89376a  polymake: Update info on macOS

0c1b044  polymake: update MongoDB information for linux distros

e46cd38  Merge branch 't/24905/public/upgrade_polymake_to_version_3_2r4' into t/22704/public/22704

comment:28 Changed 2 years ago by
does this need #24905 ?
comment:29 Changed 2 years ago by
I did this work on top of the #24905 branch because otherwise there would be big merge conflicts.
comment:30 followup: ↓ 32 Changed 2 years ago by
that's what I get after trying this on Linux:
$ ./sage tp optional=bliss,build,dochtml,e_antic,gap_packages,jupymake,lidia,lrslib,memlimit,ninja_build,normaliz,python2,polymake,sage src/sage/interfaces/polymake.py too many failed tests, not using stored timings Running doctests with ID 2019050822081711941b32. Git branch: testwip Using optional=bliss,build,dochtml,e_antic,gap_packages,jupymake,lidia,lrslib,memlimit,ninja_build,normaliz,polymake,python2,sage Doctesting 1 file using 4 threads. sage t src/sage/interfaces/polymake.py ********************************************************************** File "src/sage/interfaces/polymake.py", line 765, in sage.interfaces.polymake.PolymakeAbstract.application Failed example: polymake.eval('application "killerapp";') # optional  polymake Expected: Traceback (most recent call last): ... PolymakeError: Unknown application killerapp Got: <BLANKLINE> Traceback (most recent call last): File "/mnt/opt/Sage/sagedev/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 671, in _run self.compile_and_execute(example, compiler, test.globs) File "/mnt/opt/Sage/sagedev/local/lib/python2.7/sitepackages/sage/doctest/forker.py", line 1095, in compile_and_execute exec(compiled, globs) File "<doctest sage.interfaces.polymake.PolymakeAbstract.application[15]>", line 1, in <module> polymake.eval('application "killerapp";') # optional  polymake File "/mnt/opt/Sage/sagedev/local/lib/python2.7/sitepackages/sage/interfaces/polymake.py", line 2520, in eval raise PolymakeError(error) PolymakeError: Can't locate object method "compile_scope" via package "Polymake::Core::Shell::Mock" at /mnt/opt/Sage/sagedev/local/share/polymake/perllib/Polymake/utils.pl line 247. <BLANKLINE> ********************************************************************** 1 item had failures: 1 of 17 in sage.interfaces.polymake.PolymakeAbstract.application [317 tests, 1 failure, 9.54 s]  sage t src/sage/interfaces/polymake.py # 1 doctest failed 
comment:31 followup: ↓ 33 Changed 2 years ago by
and also
********************************************************************** File "src/sage/geometry/polyhedron/base.py", line 175, in sage.geometry.polyhedron.base.Polyhedron_base._sage_input_ Failed example: P = Polyhedron(vertices = [[1, 0], [0, 1]], rays = [[1, 1]], backend='polymake') # optional  polymake Expected nothing Got: gmp v.6.1 ... File "src/sage/geometry/polyhedron/backend_polymake.py", line 59, in sage.geometry.polyhedron.backend_polymake.Polyhedron_polymake Failed example: p = Polyhedron(vertices=[(0,0),(1,0),(0,1)], rays=[(1,1)], # optional  polymake lines=[], backend='polymake') Expected nothing Got: gmp v.6.1
comment:32 in reply to: ↑ 30 Changed 2 years ago by
Replying to dimpase:
that's what I get after trying this on Linux:
$ ./sage tp optional=bliss,build,dochtml,e_antic,gap_packages,jupymake,lidia,lrslib,memlimit,ninja_build,normaliz,python2,polymake,sage src/sage/interfaces/polymake.py .... raise PolymakeError(error) PolymakeError: Can't locate object method "compile_scope" via package "Polymake::Core::Shell::Mock" at /mnt/opt/Sage/sagedev/local/share/polymake/perllib/Polymake/utils.pl line 247.
Yes, I know this one  see ticket description. I have reported it upstream.
comment:33 in reply to: ↑ 31 ; followup: ↓ 37 Changed 2 years ago by
Replying to dimpase:
and also
********************************************************************** File "src/sage/geometry/polyhedron/base.py", line 175, in sage.geometry.polyhedron.base.Polyhedron_base._sage_input_ Failed example: P = Polyhedron(vertices = [[1, 0], [0, 1]], rays = [[1, 1]], backend='polymake') # optional  polymake Expected nothing Got: gmp v.6.1 ... File "src/sage/geometry/polyhedron/backend_polymake.py", line 59, in sage.geometry.polyhedron.backend_polymake.Polyhedron_polymake Failed example: p = Polyhedron(vertices=[(0,0),(1,0),(0,1)], rays=[(1,1)], # optional  polymake lines=[], backend='polymake') Expected nothing Got: gmp v.6.1
Looks like this message comes from lrslib, I'll investigate
comment:34 followup: ↓ 36 Changed 2 years ago by
Actually on my machine polymake does not find the installed lrslib (it's looking for liblrs but we only have liblrsgmp.)
comment:35 Changed 2 years ago by
I also only have liblrsgmp
. Not sure if the following helps, but still:
$ ldd local/lib/libpolymake.so linuxvdso.so.1 (0x00007fff97bbb000) libperl.so.5.28 => /usr/lib64/libperl.so.5.28 (0x00007fcae6d94000) libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00007fcae6bef000) libdl.so.2 => /lib64/libdl.so.2 (0x00007fcae6be9000) libmpfr.so.6 => /mnt/opt/Sage/sagedev/local/lib/libmpfr.so.6 (0x00007fcae6b6a000) libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007fcae68df000) libpthread.so.0 => /lib64/libpthread.so.0 (0x00007fcae68bc000) libstdc++.so.6 => /usr/lib/gcc/x86_64pclinuxgnu/9.1.0/libstdc++.so.6 (0x00007fcae6642000) libm.so.6 => /lib64/libm.so.6 (0x00007fcae6505000) libgcc_s.so.1 => /usr/lib/gcc/x86_64pclinuxgnu/9.1.0/libgcc_s.so.1 (0x00007fcae64eb000) libc.so.6 => /lib64/libc.so.6 (0x00007fcae631b000) libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007fcae62e2000) libicuuc.so.64 => /usr/lib64/libicuuc.so.64 (0x00007fcae60f1000) libz.so.1 => /lib64/libz.so.1 (0x00007fcae5ecf000) /lib64/ldlinuxx8664.so.2 (0x00007fcae71bd000) libicudata.so.64 => /usr/lib64/libicudata.so.64 (0x00007fcae4489000)
the rest of polymake libs are statically linked, so it's not as easy to gather whether lrs has anything to do with it.
comment:36 in reply to: ↑ 34 Changed 2 years ago by
comment:37 in reply to: ↑ 33 Changed 2 years ago by
Replying to mkoeppe:
Replying to dimpase:
and also
********************************************************************** File "src/sage/geometry/polyhedron/base.py", line 175, in sage.geometry.polyhedron.base.Polyhedron_base._sage_input_ Failed example: P = Polyhedron(vertices = [[1, 0], [0, 1]], rays = [[1, 1]], backend='polymake') # optional  polymake Expected nothing Got: gmp v.6.1 ... File "src/sage/geometry/polyhedron/backend_polymake.py", line 59, in sage.geometry.polyhedron.backend_polymake.Polyhedron_polymake Failed example: p = Polyhedron(vertices=[(0,0),(1,0),(0,1)], rays=[(1,1)], # optional  polymake lines=[], backend='polymake') Expected nothing Got: gmp v.6.1Looks like this message comes from lrslib, I'll investigate
After the lrs fix on #27803, one of these 2 errors is still present:
File "src/sage/geometry/polyhedron/base.py", line 175, in sage.geometry.polyhedron.base.Polyhedron_base._sage_input_ Failed example: P = Polyhedron(vertices = [[1, 0], [0, 1]], rays = [[1, 1]], backend='polymake') # optional  polymake Expected nothing Got: gmp v.6.1
comment:38 followup: ↓ 39 Changed 2 years ago by
Yes, and with #27803 I can reproduce it. I think these errors appear nondeterministicaly.
comment:39 in reply to: ↑ 38 ; followup: ↓ 41 Changed 2 years ago by
Replying to mkoeppe:
Yes, and with #27803 I can reproduce it. I think these errors appear nondeterministicaly.
I think it is deterministic, and occurs on the 1st invocation of the polymake interface. Try starting Sage and running
P = Polyhedron(vertices = [[1, 0], [0, 1]], rays = [[1, 1]], backend='polymake')
repeatedly.
comment:40 Changed 2 years ago by
 Dependencies changed from #24905 to #24905, #27803
comment:41 in reply to: ↑ 39 Changed 2 years ago by
Replying to dimpase:
I think it is deterministic, and occurs on the 1st invocation of the polymake interface.
You are right.
comment:42 followup: ↓ 43 Changed 2 years ago by
The new behavior from this ticket comes from the fact that with the new inprocess polymake, lrslib prints directly to stdout, so this output is not captured.
Perhaps lrslib should be configured with LRS_QUIET
.
comment:43 in reply to: ↑ 42 Changed 2 years ago by
Replying to mkoeppe:
Perhaps lrslib should be configured with
LRS_QUIET
.
lrslib 070 removes this output unconditionally.
comment:44 Changed 2 years ago by
 Dependencies changed from #24905, #27803 to #24905, #27803, #27805
comment:45 Changed 2 years ago by
 Commit changed from e46cd380e9645b0e7e6a26474aa2f69999f7cdc3 to c84d89d552396a83638ee7a95de77c1a01c54e58
Branch pushed to git repo; I updated commit sha1. New commits:
2eda6e1  polymake: add to fedora instructions

5916141  Fix polymake lrslib detection

086bb2d  Merge branch 't/27803/public/27803polymakelrslibdetection' into t/22704/public/22704

1b47911  Configure lrslib with LRS_QUIET

c84d89d  Merge branch 't/27805/configure_lrslib_with_lrs_quiet' into t/22704/public/22704

comment:46 Changed 2 years ago by
comment:47 Changed 2 years ago by
 Dependencies changed from #24905, #27803, #27805 to #24905
 Reviewers set to Dima Pasechnik
 Status changed from needs_review to positive_review
OK, this is fixed by #27805 indeed.
New commits:
2eda6e1  polymake: add to fedora instructions

5916141  Fix polymake lrslib detection

086bb2d  Merge branch 't/27803/public/27803polymakelrslibdetection' into t/22704/public/22704

1b47911  Configure lrslib with LRS_QUIET

c84d89d  Merge branch 't/27805/configure_lrslib_with_lrs_quiet' into t/22704/public/22704

comment:48 Changed 2 years ago by
 Description modified (diff)
I've put the remaining minor issue to a followup ticket, #27807 (Polymakejupymake interface: Fix polymake's error handling in Shell::Mock)
comment:49 Changed 2 years ago by
Should I make that last failing doctest less specific regarding the error message for this ticket? (Before it's fixed fully in #27807)
comment:50 Changed 2 years ago by
If you like, you can tag it "known error".
comment:51 Changed 2 years ago by
 Commit changed from c84d89d552396a83638ee7a95de77c1a01c54e58 to 58aa1ad052d7c2901b919cac9863d8d867d574c1
 Status changed from positive_review to needs_review
Branch pushed to git repo; I updated commit sha1 and set ticket back to needs_review. New commits:
58aa1ad  Mark issue of #27807 as known bug

comment:52 Changed 2 years ago by
 Status changed from needs_review to positive_review
comment:54 Changed 2 years ago by
 Commit changed from 58aa1ad052d7c2901b919cac9863d8d867d574c1 to 8d9e2c26686c4bbb0fac7d9b30fd60a11ef37446
Branch pushed to git repo; I updated commit sha1. New commits:
8d9e2c2  Merge tag '8.8.beta5' into t/22704/public/22704

comment:55 Changed 2 years ago by
 Status changed from needs_work to positive_review
comment:56 Changed 2 years ago by
 Branch changed from public/22704 to 8d9e2c26686c4bbb0fac7d9b30fd60a11ef37446
 Resolution set to fixed
 Status changed from positive_review to closed
comment:57 Changed 2 years ago by
 Commit 8d9e2c26686c4bbb0fac7d9b30fd60a11ef37446 deleted
The upstream package jupymake seems to be missing on the sage mirrors
comment:58 followup: ↓ 59 Changed 2 years ago by
Do I understand correctly that in the current version jupymake is necessary for polymake to function properly? Shouldn't jupymake become a dependancy of polymake?
comment:59 in reply to: ↑ 58 Changed 2 years ago by
Replying to jipilab:
Do I understand correctly that in the current version jupymake is necessary for polymake to function properly? Shouldn't jupymake become a dependancy of polymake?
No, there are two interface classes, which both work. PolymakeExpect
(aliases to Polymake
) and PolymakeJupyMake
.
Depending on whether you have jupymake, polymake
is set to an instance of one or the other.
Here's a first version.
Last 10 new commits:
polymake doctest: Mark a doctest random that depends on installed packages
Polymake._tab_completion: Fix for new apropos format in Polymake 3.4
Doctest fix for polymake 3.4: false/true instead of 0/1
Doctest fix for polymake 3.4: Use different function from the tropical application
Doctest fix for polymake 3.4: Differences in available properties
Polymake 3.4: Fix for typeof small objects
Polymake: Remove copy/paste error handling for code that cannot raise
polymake: Make normaliz a dependency
polymake: Add a patch to polymake's libnormaliz configure
Refactor interface Polymake through PolymakeAbstract, add PolymakeJuPyMake