spkg-configure.m4 for polymake

[mkoeppe]

Polymake is available on a number of systems (​https://repology.org/project/polymake/versions) and also on Homebrew via a custom tap: ​https://polymake.org/doku.php/download/start#binary_releases

We add system package info and spkg-configure.m4

Fixing broken distribution packages is beyond the scope of the ticket.

[mkoeppe]

New commits:

​40996b3	.homebrew-build-env: Use /usr/local/opt/polymake if it exists
​2bbee93	build/pkgs/polymake/spkg-configure.m4: New

[mkoeppe]

With polymake from Andreas's tap, need to do export PERL5LIB="/usr/local/Cellar/polymake/4.3_1/libexec/perl5/lib/perl5/darwin-thread-multi-2level:/usr/local/Cellar/polymake/4.3_1/libexec/perl5/lib/perl5" (this is done in the polymake script but we use the callable library through jupymake)

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​00b3109	build/pkgs/sagelib/spkg-install: Set SETUPTOOLS_USE_DISTUTILS=local
​fd23c0d	build/pkgs/setuptools/patches: Add distutils patch from https://github.com/fxcoudert/cpython/commit/6511bf56.patch
​3601c51	Adapt the patch to apply to the vendored distutils
​9ad84cc	Merge branch 't/31344/homebrew__docbuild_crashes__libtcl_atforkprepare' into t/31335/homebrew__unused_packages__singular__pari_______in__usr_local_leak_into_sagelib_build
​ae4ea55	Merge branch 't/31344/homebrew__docbuild_crashes__libtcl_atforkprepare' into t/31335/homebrew__unused_packages__singular__pari_______in__usr_local_leak_into_sagelib_build
​add7db4	Merge tag '9.3.beta7' into t/31335/homebrew__unused_packages__singular__pari_______in__usr_local_leak_into_sagelib_build
​4e777e7	Merge tag '9.3.beta8' into t/31335/homebrew__unused_packages__singular__pari_______in__usr_local_leak_into_sagelib_build
​e03c5ac	build/bin/sage-build-env: Set SETUPTOOLS_USE_DISTUTILS here, not in build/pkgs/sagelib/spkg-install
​f125453	Merge #31335
​cd792c5	build/pkgs/polymake/distros: Add more

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​55211fa	build/pkgs/latte_int: Upgrade to 1.7.6
​78daa99	build/pkgs/latte_int/dependencies: Make lrslib only an optional dependency
​156fc9f	Merge #31482
​4fef65a	build/pkgs/polymake/dependencies: Make lrslib only an optional dependency

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​35558b3

build/pkgs/polymake/spkg-install.in: Only pass --with-lrs if it is installed

[mkoeppe]

Replying to mkoeppe:

With polymake from Andreas's tap, need to do export PERL5LIB="/usr/local/Cellar/polymake/4.3_1/libexec/perl5/lib/perl5/darwin-thread-multi-2level:/usr/local/Cellar/polymake/4.3_1/libexec/perl5/lib/perl5" (this is done in the polymake script but we use the callable library through jupymake)

Setting this globally unfortunately breaks autotools (aclocal) from homebrew:

Usage: DynaLoader::dl_find_symbol(libhandle, symbolname) at /usr/local/Cellar/polymake/4.3_1/libexec/perl5/lib/perl5/darwin-thread-multi-2level/XSLoader.pm line 89.
Compilation failed in require at /usr/local/Cellar/polymake/4.3_1/libexec/perl5/lib/perl5/File/Path.pm line 6.
BEGIN failed--compilation aborted at /usr/local/Cellar/polymake/4.3_1/libexec/perl5/lib/perl5/File/Path.pm line 6.
Compilation failed in require at /usr/local/bin/aclocal line 33.
BEGIN failed--compilation aborted at /usr/local/bin/aclocal line 33.

Too bad that polymake-config does not advertise the necessary Perl library path...

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​93fcbd6

build/pkgs/polymake/distros/debian.txt: Add libpolymake-dev

[mkoeppe]

Moving this ticket to 9.4, as it seems unlikely that it will be merged in 9.3, which is in the release candidate stage

[mkoeppe]

Saving the continuation:

JuPyMake/JuPyMake?.cpp at master · sebasguts/JuPyMake ​https://github.com/sebasguts/JuPyMake/blob/master/JuPyMake.cpp

jupyter-polymake/kernel.py at master · polymake/jupyter-polymake ​https://github.com/polymake/jupyter-polymake/blob/master/jupyter_kernel_polymake/kernel.py

jupyter-kernel-polymake · PyPI ​https://pypi.org/project/jupyter-kernel-polymake/

Failed to install on OSX Mojave Mac::SystemDirectory? is not installed. · Issue #39 · sqitchers/homebrew-sqitch ​https://github.com/sqitchers/homebrew-sqitch/issues/39

tokuhirom/plenv: Perl binary manager ​https://github.com/tokuhirom/plenv

polymake Downloads [polymake wiki] ​https://polymake.org/doku.php/download/start

[mkoeppe]

Perlbrew ​https://perlbrew.pl/

gugod/App-perlbrew: Manage perl installations in your $HOME ​https://github.com/gugod/App-perlbrew

App::perlbrew - Manage perl installations in your $HOME - metacpan.org ​https://metacpan.org/pod/App::perlbrew

polymake Downloads [polymake wiki] ​https://polymake.org/doku.php/download/start#binary_releases

perl:mongodb package versions - Repology ​https://repology.org/project/perl:mongodb/versions

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​b211f2a	.homebrew-build-env: Use /usr/local/opt/polymake if it exists
​27b4992	build/pkgs/polymake/spkg-configure.m4: New
​d30256b	build/pkgs/polymake/spkg-configure.m4: If system polymake is in use, do not require Perl packages
​6860f90	build/pkgs/polymake/distros: Add more
​f4eb33b	build/pkgs/polymake/dependencies: Make lrslib only an optional dependency
​d93788a	build/pkgs/polymake/spkg-install.in: Only pass --with-lrs if it is installed
​9ad9e2f	build/pkgs/polymake/distros/debian.txt: Add libpolymake-dev

[mkoeppe]

Replying to mkoeppe:

With polymake from Andreas's tap, need to do export PERL5LIB="[...]" (this is done in the polymake script but we use the callable library through jupymake)

Fixed in latest version of the homebrew formula according to ​https://github.com/polymake/polybundle/issues/19#issuecomment-938612849

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​2576f86	build/make/Makefile.in: If a script package has no spkg-install, run "sage -info" and exit with error
​feb8de7	build/pkgs/: Remove spkg-install scripts for dummy script packages
​8f782c0	.github/workflows/tox-{optional,experimental}.yml: Do not try to test dummy script packages
​4b292be	build/pkgs/perl_mongodb/spkg-install: Remove
​a7b6352	build/bin/sage-spkg-info: Fix display of system packages
​e65b309	bootstrap: Do not provide ./configure --enable-SPKG options for dummy optional packages
​b485d46	m4/sage_spkg_collect.m4: Do not advertise dummy optional packages as installable
​8c8e1c4	Merge #31163
​015b899	build/bin/sage-print-system-package-command: Handle cpan
​1832397	Merge #32605

[git]

Branch pushed to git repo; I updated commit sha1. Last 10 new commits:

​ea548d7	sage.features.four_ti_2: New, use it in sage.interfaces.four_ti_2, sage.sandpiles
​f826ded	build/pkgs/4ti2/spkg-configure.m4: Check for executable's with prefix 4ti2_ too
​56016ce	use AC_LINK_IFELSE instead of obsolete AC_TRY_LINK
​2b45b77	sage.feature.join_feature: New, factored out from LatteFeature; use it to implement FourTi2Feature
​5c23cc9	DocTestReporter: Fix 'sage -t --optional=all'
​1b8634d	sage.doctest.external: Add 4ti2
​d9d4f99	Merge tag '9.4.beta6' into t/30887/public/30887
​646e182	src/sage/features/four_ti_2.py: Move import of SAGE_ENV inside the __init__ method, to remove confusion of sage.misc.dev_tools
​b090dc0	Merge #30887
​83a98be	src/sage/features/polymake.py: New

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ccd779a	#32547 : replace nonexistent llvm-toolchain with clang as Debian alternative to llvm spkg
​b2281fa	Merge #32547
​d2cbfe7	build/pkgs/polymake/distros/homebrew.txt: New

[mkoeppe]

Tested successfully on homebrew

[mkoeppe]

Testing with tox -e docker-ubuntu-groovy-maximal, where configure accepts system polymake 4.1-4build1, gives many doctest failures of the form:

      File "/sage/local/lib/python3.8/site-packages/sage/interfaces/polymake.py", line 2598, in _start
        InitializePolymake()          # Can only be called once
    JuPyMake.PolymakeError: polymake::Main - /usr/lib/polymake/shared is not a symlink

Indeed:

ls -l /usr/lib/polymake/
total 12
-rw-r--r-- 1 root root 2024 Aug 17  2020 config.ninja
drwxr-xr-x 2 root root 4096 Oct  8 21:18 lib
drwxr-xr-x 3 root root 4096 Oct  8 21:18 perlx

[mkoeppe]

... so clearly we need to extend spkg-configure.m4 to test that libpolymake has not been debianized to death

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​f1f362d	build/pkgs/polymake/spkg-configure.m4: Actually use POLYMAKE_VERSION_MIN
​c6b5f8c	build/pkgs/polymake/spkg-configure.m4: Link and run a test program with libpolymake

[mkoeppe]

Now correctly rejects polymake on this system (tox -e docker-ubuntu-groovy-maximal -- config.status)

[mkoeppe]

Same on ubuntu-impish-maximal (with system polymake 4.3)

[mkoeppe]

On fedora-34-maximal:

configure:34443: checking whether libpolymake works
configure:34479: g++ -o conftest  -I/usr/include/polymake/external -I/usr/include/arb -I/usr/include/eigen3 -I/usr/include/gfanlib -I/usr/include/nauty -I/usr/include -I/usr/include -DPOLYMAKE_VERSION=404 -fPIC -pipe -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wno-unused-local-typedefs -std=c++14 -ftemplate-depth-200 -fno-strict-aliasing -fopenmp -Wshadow -Wlogical-op -Wconversion -Wzero-as-null-pointer-constant -Wno-parentheses -Wno-error=unused-function -Wno-stringop-overflow -Wno-array-bounds -Wno-maybe-uninitialized -Wno-free-nonheap-object -DPOLYMAKE_WITH_FLINT -O2 -DPOLYMAKE_DEBUG=0   -Dlinux   -I/usr/include -lpolymake -Wl,-z,relro -Wl,--as-needed -lnormaliz -ldl -fuse-ld=gold -fopenmp -L/usr/lib64 -Wl,-E  conftest.cpp -lmpfi -lmpc -lm4rie -llrcalc -lnauty -lhomfly -liml -lgiac -lecm -lpari -lcurl -lcliquer -lcddgmp -lbz2 -lflint -lmpfr -lglpk -lgmp -lm  -lntl -lLfunction >&5
cc1plus: error: '-Wformat-security' ignored without '-Wformat' [-Werror=format-security]
cc1plus: some warnings being treated as errors

[mkoeppe]

Same on fedora-35-maximal

So system polymake is rejected. This is correct because building jupymake would fail with the same error.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​d9fc03e

build/pkgs/perl_cpan_polymake_prereq/distros/fedora.txt: perl-TermReadKey, not perl-Term-ReadKey

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​777d1b8

build/pkgs/polymake/distros/nix.txt: New

[dimpase]

needs a rebase

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​edce768

Merge tag '9.5.beta3' into t/31527/spkg_configure_m4_for_polymake

[dimpase]

gentoo's polymake is too old atm, it's 3.0_p2 there (and broken).

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​717f98b

build/pkgs/polymake/distros/gentoo.txt: Remove

[mkoeppe]

Let's get this in please

[dimpase]

fails to recognize polymake on Fedora 34 with polymake package installed, due to -Werror=format-security in g++ call from ./configure.

configure:36908: checking for polymake-config >= 3.5
configure:36984: result: /usr/bin/polymake-config
configure:36989: checking whether libpolymake works
configure:37025: g++ -o conftest  -I/usr/include/polymake/external -I/usr/include/arb -I/usr/include/eigen3 -I/usr/include/gfanlib -I/usr/include/nauty -I/usr/include -I/usr/include -DPOLYMAKE_VERSION=404 -fPIC -pipe -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wno-unused-local-typedefs -std=c++14 -ftemplate-depth-200 -fno-strict-aliasing -fopenmp -Wshadow -Wlogical-op -Wconversion -Wzero-as-null-pointer-constant -Wno-parentheses -Wno-error=unused-function -Wno-stringop-overflow -Wno-array-bounds -Wno-maybe-uninitialized -Wno-free-nonheap-object -DPOLYMAKE_WITH_FLINT -O2 -DPOLYMAKE_DEBUG=0   -Dlinux   -I/usr/include -lpolymake -Wl,-z,relro -Wl,--as-needed -lnormaliz -ldl -fuse-ld=gold -fopenmp -L/usr/lib64 -Wl,-E  conftest.cpp -lmpfi -lmpc -lm4rie -llrcalc -lnauty -lhomfly -liml -lgiac -lecm -lpari -lcurl -lcliquer -lcddgmp -lbz2 -lflint -lmpfr -lglpk -lgmp -lm  -lntl -lLfunction >&5
cc1plus: error: '-Wformat-security' ignored without '-Wformat' [-Werror=format-security]
cc1plus: some warnings being treated as errors

And Werror gets there from polymake-config --cflags call.

$ polymake-config --cflags
-DPOLYMAKE_VERSION=404 -fPIC -pipe -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wno-unused-local-typedefs -std=c++14 -ftemplate-depth-200 -fno-strict-aliasing -fopenmp -Wshadow -Wlogical-op -Wconversion -Wzero-as-null-pointer-constant -Wno-parentheses -Wno-error=unused-function -Wno-stringop-overflow -Wno-array-bounds -Wno-maybe-uninitialized -Wno-free-nonheap-object -DPOLYMAKE_WITH_FLINT -O2 -DPOLYMAKE_DEBUG=0

[dimpase]

If I clean up CPPFLAGS as follows

build/pkgs/polymake/spkg-configure.m4

@@ -20 +20 @@
            saved_LDFLAGS="$LDFLAGS"
            CXX="$($ac_cv_path_POLYMAKE_CONFIG --cc)"
            CPPFLAGS="$($ac_cv_path_POLYMAKE_CONFIG --includes) $($ac_cv_path_POLYMAKE_CONFIG --cflags) $CPPFLAGS"
+           CPPFLAGS=${CPPFLAGS//\-Werror=format\-security/}
            LDFLAGS="-lpolymake $($ac_cv_path_POLYMAKE_CONFIG --ldflags) $LDFLAGS"
            AC_RUN_IFELSE([AC_LANG_PROGRAM([[
                #include <polymake/Main.h>

then I still get

configure:36989: checking whether libpolymake works
configure:37026: g++ -o conftest  -I/usr/include/polymake/external -I/usr/include/arb -I/usr/include/eigen3 -I/usr/include/gfanlib -I/usr/include/nauty -I/usr/include -I/usr/include -DPOLYMAKE_VERSION=404 -fPIC -pipe -O2 -flto=auto -ffat-lto-objects -fexceptions -g -grecord-gcc-switches -pipe  -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -Wno-unused-local-typedefs -std=c++14 -ftemplate-depth-200 -fno-strict-aliasing -fopenmp -Wshadow -Wlogical-op -Wconversion -Wzero-as-null-pointer-constant -Wno-parentheses -Wno-error=unused-function -Wno-stringop-overflow -Wno-array-bounds -Wno-maybe-uninitialized -Wno-free-nonheap-object -DPOLYMAKE_WITH_FLINT -O2 -DPOLYMAKE_DEBUG=0   -Dlinux   -I/usr/include -lpolymake -Wl,-z,relro -Wl,--as-needed -lnormaliz -ldl -fuse-ld=gold -fopenmp -L/usr/lib64 -Wl,-E  conftest.cpp -lmpfi -lmpc -lm4rie -llrcalc -lnauty -lhomfly -liml -lgiac -lecm -lpari -lcurl -lcliquer -lcddgmp -lbz2 -lflint -lmpfr -lglpk -lgmp -lm  -lntl -lLfunction >&5
configure:37026: $? = 0
configure:37026: ./conftest
Can't locate Polymake/Main.pm in @INC (you may need to install the Polymake::Main module) (@INC contains: /share/polymake/perllib /lib64/polymake/perlx/5.32.1/x86_64-linux-thread-multi /lib64/polymake/perlx/5.32.1 /lib64/polymake/perlx /users/dimpase/perl5/lib/perl5/x86_64-linux-thread-multi /users/dimpase/perl5/lib/perl5 /usr/local/lib64/perl5/5.32 /usr/local/share/perl5/5.32 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at /builddir/build/BUILD/polymake-4.4/lib/callable/src/perl/Main.cc line 146.
BEGIN failed--compilation aborted at /builddir/build/BUILD/polymake-4.4/lib/callable/src/perl/Main.cc line 146.
terminate called after throwing an instance of 'std::runtime_error'
  what():  could not initialize the perl interpreter
./configure: line 3495: 1808668 Aborted                 (core dumped) ./conftest$ac_exeext
configure:37026: $? = 134
configure: program exited with status 134

[mkoeppe]

Replying to dimpase:

fails to recognize polymake on Fedora 34 with polymake package installed, due to -Werror=format-security in g++ call from ./configure.

Yes, see comment:33.

[dimpase]

Is there any Linux distro with working for the purposes of this ticket Polymake package?

[mkoeppe]

I don't remember if I tested with arch, opensuse, or nix.

[arojas]

Works fine on Arch. Picks up system polymake, Jupymake spkg installs correctly and all tests pass in interfaces.polymake

[mkoeppe]

Thanks!

[gh-sheerluck]

Replying to dimpase:

Is there any Linux distro with working for the purposes of this ticket Polymake package?

Gentoo with ​https://github.com/gentoo/gentoo/pull/17925

sage -t --long --random-seed=0 src/sage/interfaces/polymake.py
    [54 tests, 0.63 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 0.7 seconds
    cpu time: 0.1 seconds
    cumulative wall time: 0.6 seconds
Traceback (most recent call last):
  File "/usr/lib/python-exec/python3.10/sage-runtests", line 159, in <module>
    exit_code_pytest = pytest.main(pytest_options + args)
TypeError: can only concatenate list (not "Namespace") to list

[mkoeppe]

Excellent!

[dimpase]

The present branch appears to need polymake explictly listed in the list --optional=, if it's not installed. On arch I have:

$ ./sage -tp  src/sage/geometry/polyhedron/backend_polymake.py
Running doctests with ID 2021-10-22-08-52-35-89300609.
Git branch: wip
Using --optional=arch,argcomplete,build,dochtml,jupymake,pip,sage,sage.rings.real_double,sage_spkg
Doctesting 1 file using 4 threads.
sage -t --warn-long 105.2 --random-seed=111795974516996210666725135263478181348 src/sage/geometry/polyhedron/backend_polymake.py
    [9 tests, 0.52 s]
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------
Total time for all tests: 0.6 seconds
    cpu time: 0.5 seconds
    cumulative wall time: 0.5 seconds
Pytest is not installed, skip checking tests that rely on it.

but

$ ./sage -tp --optional=arch,argcomplete,build,dochtml,pip,sage,sage.rings.real_double,sage_spkg,polymake src/sage/geometry/polyhedron/backend_polymake.py
Running doctests with ID 2021-10-22-08-52-45-3599c25d.
Git branch: wip
Using --optional=arch,argcomplete,build,dochtml,pip,polymake,sage,sage.rings.real_double,sage_spkg
Doctesting 1 file using 4 threads.
sage -t --warn-long 105.2 --random-seed=65107005796673777637867553513561507930 src/sage/geometry/polyhedron/backend_polymake.py
**********************************************************************
File "src/sage/geometry/polyhedron/backend_polymake.py", line 665, in sage.geometry.polyhedron.backend_polymake.Polyhedron_polymake.__setstate__
Failed example:
    print("Possible output"); P = polytopes.dodecahedron(backend='polymake')  # optional - polymake
Exception raised:
    Traceback (most recent call last):
      File "/home/dima/sagetrac-mirror/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/interfaces/interface.py", line 732, in __init__
        self._name = parent._create(value, name=name)
      File "/home/dima/sagetrac-mirror/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/interfaces/polymake.py", line 549, in _create
        self.set(name, value)
      File "/home/dima/sagetrac-mirror/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/interfaces/polymake.py", line 638, in set
        self.eval(cmd)
      File "/home/dima/sagetrac-mirror/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/interfaces/polymake.py", line 2712, in eval
        raise PolymakeError(error)
    sage.interfaces.polymake.PolymakeError: C++/perl Interface module compilation failed; most likely due to a type mismatch.
    Set the variable $Polymake::User::Verbose::cpp to a positive value and repeat for more details.


    During handling of the above exception, another exception occurred:
...
   TypeError: C++/perl Interface module compilation failed; most likely due to a type mismatch.
    Set the variable $Polymake::User::Verbose::cpp to a positive value and repeat for more details.

**********************************************************************
1 item had failures:
   1 of  21 in sage.geometry.polyhedron.backend_polymake.Polyhedron_polymake.__setstate__
    [106 tests, 1 failure, 13.32 s]
----------------------------------------------------------------------
sage -t --warn-long 105.2 --random-seed=65107005796673777637867553513561507930 src/sage/geometry/polyhedron/backend_polymake.py  # 1 doctest failed

there is a similar test failure in src/sage/geometry/polyhedron/base.py

[dimpase]

We see that tests mentioned in comment:54 were not complete, as polymake was not in --optional= - if it were, we'd see many more tests (this is on Arch, again):

$ ./sage -tp --optional=arch,argcomplete,build,dochtml,pip,sage,sage.rings.real_double,sage_spkg,polymake src/sage/interfaces/polymake.py                 
Running doctests with ID 2021-10-22-08-58-24-8cc8a0d8.
Git branch: wip
Using --optional=arch,argcomplete,build,dochtml,pip,polymake,sage,sage.rings.real_double,sage_spkg
Doctesting 1 file using 4 threads.
sage -t --warn-long 105.2 --random-seed=106731483408083137921384584724619392983 src/sage/interfaces/polymake.py
    [253 tests, 5.28 s]
----------------------------------------------------------------------
All tests passed!

[dimpase]

jupymake should be in build/pkgs/polymake/dependencies, as polymake tests fail if it's not installed.

As well, spkg-configure.m4 perhaps needs to at least print a warning if system's polymake it selected, or, better, use some (missing?) mechanism to trigger installation of jupymake in this case.

[gh-sheerluck]

Replying to dimpase:

we'd see many more tests (this is on Arch, again):

[253 tests, 5.28 s]

yes, with that long --optional= I see all 253 tests passed too (on Gentoo)

[mkoeppe]

Replying to dimpase:

jupymake should be in build/pkgs/polymake/dependencies, as polymake tests fail if it's not installed.

No, this makes no sense. jupymake depends (at compile time) on polymake.

[mkoeppe]

Replying to dimpase:

use some (missing?) mechanism to trigger installation of jupymake in this case.

This is a valid wishlist item.

[dimpase]

then tag jupymake-dependent tests as such.

[mkoeppe]

The failure in comment:56,

   TypeError: C++/perl Interface module compilation failed; most likely due to a type mismatch.
    Set the variable $Polymake::User::Verbose::cpp to a positive value and repeat for more details.

has got nothing to do with jupymake vs. the polymake pexpect interface.

Looks to me like a packaging bug in arch.

[mkoeppe]

Replying to mkoeppe:

Replying to dimpase:

use some (missing?) mechanism to trigger installation of jupymake in this case.

This is a valid wishlist item.

I have opened #32741 for this.

[mjo]

Replying to dimpase:

gentoo's polymake is too old atm, it's 3.0_p2 there (and broken).

I've just updated it to polymake-4.5. Send bug reports if it's still broken.

[dimpase]

Replying to mkoeppe:

Replying to mkoeppe:

Replying to dimpase:

use some (missing?) mechanism to trigger installation of jupymake in this case.

This is a valid wishlist item.

I have opened #32741 for this.

I explained there to how to fix this in this particular case - with jupymake getting spkg-configure.m4

[gh-sheerluck]

I hope #32741 is not preventing this ticket to get positive review. I mean all 253 tests passed on several systems.

[dimpase]

well, the reason for not giving it a positive review are doctests failing due to accidentally not having jupymake installed, which is not forced now.

[mkoeppe]

Dima, comment:63 -- since you tested on arch, could you please investigate the failure mode? These doctests are NOT dependent on jupymake.

[mkoeppe]

Let's move this forward please

[dimpase]

Replying to mjo:

Replying to dimpase:

gentoo's polymake is too old atm, it's 3.0_p2 there (and broken).

I've just updated it to polymake-4.5. Send bug reports if it's still broken.

# emerge --ask polymake

These are the packages that would be merged, in order:

Calculating dependencies -

!!! Problem resolving dependencies for sci-mathematics/polymake             ... done!

!!! The ebuild selected to satisfy "polymake" has unmet requirements.
- sci-mathematics/polymake-4.5::gentoo USE="cdd flint normaliz -bliss -libpolymake -lrs -nauty -ppl -singular"

  The following REQUIRED_USE flag constraints are unsatisfied:
    exactly-one-of ( bliss nauty )

[dimpase]

Replying to mkoeppe:

Let's move this forward please

sorry, my arch lxc vm is gone (I guess I forgot to make it persistent?). Let me try "native" Gentoo instead.

[mkoeppe]

Time to work on #29283 (tox.ini: Add test environments using LXC)

[dimpase]

Replying to dimpase:

Replying to mjo:

Replying to dimpase:

gentoo's polymake is too old atm, it's 3.0_p2 there (and broken).

I've just updated it to polymake-4.5. Send bug reports if it's still broken.

USE="cdd flint normaliz libpolymake lrs nauty ppl singular" emerge --ask polymake

doesn't work, there is a problem with lrs (and Sage installs polymake with lrs) Apparently, it's known to Sage, as spkg-install.in has a comment:

# Since polymake v3.4, it does not find our lrs installation if we do not provide --with-lrs explicitly.

Once this is fixed in Gentoo (sorry, I don't grok its ebuilds), distros/gentoo.txt should be provided (with appropriate [..] options?).

[mjo]

Replying to dimpase:

  The following REQUIRED_USE flag constraints are unsatisfied:
    exactly-one-of ( bliss nauty )

You have to set USE=bliss or USE=nauty, but not both, to pick a backend for graph automorphism computations.

[dimpase]

Replying to mjo:

Replying to dimpase:

  The following REQUIRED_USE flag constraints are unsatisfied:
    exactly-one-of ( bliss nauty )

You have to set USE=bliss or USE=nauty, but not both, to pick a backend for graph automorphism computations.

sure, but the default USE flags shouldn't fail! As well, how about lrs?

[mjo]

Replying to dimpase:

sure, but the default USE flags shouldn't fail! As well, how about lrs?

They don't "fail," portage tells you to pick one because you have to pick one =P

If one is a significantly better choice than the other, we would normally enable that one by default, but I don't know enough about polymake to say that either is better.

What's wrong with lrs?

[dimpase]

USE="cdd flint normaliz libpolymake lrs nauty ppl singular" emerge --ask polymake

These are the packages that would be merged, in order:

Calculating dependencies... done!
[ebuild   R    ] sci-mathematics/polymake-4.5  USE="lrs*" 

Would you like to merge these packages? [Yes/No] 

>>> Verifying ebuild manifests

>>> Emerging (1 of 1) sci-mathematics/polymake-4.5::gentoo
 * polymake-4.5-minimal.tar.bz2 BLAKE2B SHA512 size ;-) ...            [ ok ]
>>> Unpacking source...
>>> Unpacking polymake-4.5-minimal.tar.bz2 to /var/tmp/portage/sci-mathematics/polymake-4.5/work
>>> Source unpacked in /var/tmp/portage/sci-mathematics/polymake-4.5/work
>>> Preparing source in /var/tmp/portage/sci-mathematics/polymake-4.5/work/polymake-4.5 ...
>>> Source prepared.
>>> Configuring source in /var/tmp/portage/sci-mathematics/polymake-4.5/work/polymake-4.5 ...
checking C++ compiler ... ok (x86_64-pc-linux-gnu-g++ is GCC 8.3.1)
checking C++ library ... ok (GNU libstdc++ 20190518, C++ 201402)
determining architecture ... ok (x86_64)
determining compiler flags ... ok
   CFLAGS=-march=native -O3 -fomit-frame-pointer -pipe
   CXXFLAGS=-march=native -O3 -fomit-frame-pointer -pipe -std=c++14 -ftemplate-depth-200 -fno-strict-aliasing -fopenmp -Wshadow -Wlogical-op -Wconversion -Wzero-as-null-pointer-constant -Wno-parentheses -Wno-error=unused-function
   LDFLAGS=-Wl,-O1 -Wl,--as-needed -fuse-ld=gold -fopenmp
checking gmp installation ... ok
checking mpfr installation ... ok
checking boost installation ... ok
checking perl module XML::SAX ... ok
checking perl module Term::ReadKey ... ok
checking perl module Term::ReadLine ... ok
checking perl module JSON ... ok
checking shared perl library ... ok

Configuring bundled extensions:
bundled extension java ... disabled by command-line
bundled extension javaview ... disabled by command-line
bundled extension scip ... disabled by command-line
bundled extension soplex ... disabled by command-line
bundled extension bliss ... disabled by command-line
bundled extension cdd ... ok (0.94m @ /usr)
bundled extension flint ... ok (ok (2.7.1 @ /usr))
bundled extension libnormaliz ... ok (3.8.1 @ /usr)
bundled extension lrs ... failed

ERROR:
The bundled extension lrs was explicitly requested but failed to configure.
Please recheck your argument (--with-lrs=/usr) and build/bundled.log .
You can also disable it by specifying --without-lrs instead.
 * ERROR: sci-mathematics/polymake-4.5::gentoo failed (configure phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line 127:  Called src_configure
 *   environment, line 1598:  Called die
 * The specific snippet of code:
 *       ./configure --prefix="${EPREFIX}/usr" --libdir="${EPREFIX}/usr/$(get_libdir)" --libexecdir="${EPREFIX}/usr/$(get_libdir)/polymake" $(usev !libpolymake "--without-callable") --without-java --without-javaview --without-native --without-scip --without-soplex $(use_with bliss bliss "${EPREFIX}/usr") $(use_with cdd cdd "${EPREFIX}/usr") $(use_with flint flint "${EPREFIX}/usr") $(use_with lrs lrs "${EPREFIX}/usr") $(use_with nauty nauty "${EPREFIX}/usr") $(use_with normaliz libnormaliz "${EPREFIX}/usr") $(use_with ppl ppl "${EPREFIX}/usr") $(use_with singular singular "${EPREFIX}/usr") || die
 * 
 * If you need support, post the output of `emerge --info '=sci-mathematics/polymake-4.5::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sci-mathematics/polymake-4.5::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sci-mathematics/polymake-4.5/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sci-mathematics/polymake-4.5/temp/environment'.
 * Working directory: '/var/tmp/portage/sci-mathematics/polymake-4.5/work/polymake-4.5'
 * S: '/var/tmp/portage/sci-mathematics/polymake-4.5/work/polymake-4.5'

>>> Failed to emerge sci-mathematics/polymake-4.5, Log file:

>>>  '/var/tmp/portage/sci-mathematics/polymake-4.5/temp/build.log'

 * Messages for package sci-mathematics/polymake-4.5:

 * ERROR: sci-mathematics/polymake-4.5::gentoo failed (configure phase):
 *   (no error message)
 * 
 * Call stack:
 *     ebuild.sh, line 127:  Called src_configure
 *   environment, line 1598:  Called die
 * The specific snippet of code:
 *       ./configure --prefix="${EPREFIX}/usr" --libdir="${EPREFIX}/usr/$(get_libdir)" --libexecdir="${EPREFIX}/usr/$(get_libdir)/polymake" $(usev !libpolymake "--without-callable") --without-java --without-javaview --without-native --without-scip --without-soplex $(use_with bliss bliss "${EPREFIX}/usr") $(use_with cdd cdd "${EPREFIX}/usr") $(use_with flint flint "${EPREFIX}/usr") $(use_with lrs lrs "${EPREFIX}/usr") $(use_with nauty nauty "${EPREFIX}/usr") $(use_with normaliz libnormaliz "${EPREFIX}/usr") $(use_with ppl ppl "${EPREFIX}/usr") $(use_with singular singular "${EPREFIX}/usr") || die
 * 
 * If you need support, post the output of `emerge --info '=sci-mathematics/polymake-4.5::gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sci-mathematics/polymake-4.5::gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sci-mathematics/polymake-4.5/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sci-mathematics/polymake-4.5/temp/environment'.
 * Working directory: '/var/tmp/portage/sci-mathematics/polymake-4.5/work/polymake-4.5'
 * S: '/var/tmp/portage/sci-mathematics/polymake-4.5/work/polymake-4.5'
hilbert /tmp # 

I guess one needs to do what's done in spkg-install to pass the lrs prefix more explicitly.

[mjo]

I guess one needs to do what's done in spkg-install to pass the lrs prefix more explicitly.

You may need to upgrade to lrslib-071b or lrslib-071b-r1. Those versions (I added them to the tree along with polymake-4.5) fix a critical bug in v071. I don't know if v062 works because it doesn't even build anymore. Comically, we can't remove it though because it's "stable" and no one has stabilized v071b on x86 yet.

[dimpase]

Replying to mjo:

I guess one needs to do what's done in spkg-install to pass the lrs prefix more explicitly.

You may need to upgrade to lrslib-071b or lrslib-071b-r1. Those versions (I added them to the tree along with polymake-4.5) fix a critical bug in v071. I don't know if v062 works because it doesn't even build anymore. Comically, we can't remove it though because it's "stable" and no one has stabilized v071b on x86 yet.

This does not take away the problem that polymake's configure fails, I cannot imagine it's due to lrs's version.

[dimpase]

Oops, I take it back - lrslib-071b-r1 allows configure to pass.

I don't know whether Gentoo has such a flexibility in configurations, but ideally it should refuse to even start building with an old lrs version.

[dimpase]

but I still have bad news, in Gentoo case in particular, but also in general. Perl's MongoDB module is at EOL.

# emerge dev-perl/MongoDB
Calculating dependencies... done!

!!! All ebuilds that could satisfy "dev-perl/MongoDB" have been masked.
!!! One of the following masked packages is required to complete your request:
- dev-perl/MongoDB-1.8.0::gentoo (masked by: package.mask)
/usr/portage/profiles/package.mask:
# Sam James <sam@gentoo.org> (2021-10-25)
# Fails to build with glibc-2.30(!) and was abandoned upstream
# a few years ago. No reverse dependencies in tree.
# bug #728556. Removal on 2021-11-25.


For more information, see the MASKED PACKAGES section in the emerge
man page or refer to the Gentoo Handbook.

​https://packages.gentoo.org/packages/dev-perl/MongoDB mentions version 2.2.2, and ​https://metacpan.org/dist/MongoDB says

 Changes for version v2.2.2 - 2020-08-13

    !!! END OF LIFE NOTICE !!!
        As of August 13, 2020, the MongoDB Perl driver has reached end of life and is no longer supported by MongoDB.

[dimpase]

Shouldn't perl_cpan_polymake_prereq (and other deps?) be in SAGE_SPKG_DEPCHECK for polymake?

[gh-sheerluck]

MongoDB cannot deprive this ticket of positive review

[dimpase]

We should check that polymake is working, not only polymake-config. (As some Perl deps might be broken - e.g. I ran into

$ polymake
Welcome to polymake version 4.5
Copyright (c) 1997-2021
Ewgenij Gawrilow, Michael Joswig, and the polymake team
Technische Universität Berlin, Germany
https://polymake.org

This is free software licensed under GPL; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Loading applications now...polymake:  ERROR: "/usr/share/polymake/apps/common/perllib/PolyDB.pm", line 23: Can't locate MongoDB.pm in @INC (you may need to install the MongoDB module) (@INC contains: Polymake::Core::Application=ARRAY(0x5589c39eea50) /usr/share/polymake/perllib /usr/lib64/polymake/perlx/5.34.0/x86_64-linux-thread-multi /usr/lib64/polymake/perlx/5.34.0 /usr/lib64/polymake/perlx /etc/perl /usr/local/lib64/perl5/5.34/x86_64-linux-thread-multi /usr/local/lib64/perl5/5.34 /usr/lib64/perl5/vendor_perl/5.34/x86_64-linux-thread-multi /usr/lib64/perl5/vendor_perl/5.34 /usr/lib64/perl5/5.34/x86_64-linux-thread-multi /usr/lib64/perl5/5.34 /usr/lib64/perl5/5.32 /usr/lib64/perl5/vendor_perl/5.32 /usr/lib64/perl5/vendor_perl/5.30.3 /usr/lib64/perl5/vendor_perl/5.30.0)

[dimpase]

I was able to install the missing perl module on Gentoo by doing as root

# emerge App-cpanminus
# cpanm MongoDB

this needs another ticket.

[dimpase]

Please see #32808 for Perl MongoDB package/spkg issues.

[mkoeppe]

See also #31830 (perl_cpan_polymake_prereq: Update install instructions).

[mkoeppe]

Replying to dimpase:

Shouldn't perl_cpan_polymake_prereq (and other deps?) be in SAGE_SPKG_DEPCHECK for polymake?

perl_cpan_polymake_prereq does not make sense in SAGE_SPKG_DEPCHECK because it is a dummy script package -- we provide no spkg-install for it.

You can add SAGE_SPKG_DEPCHECK for bliss, cddlib, lrslib, readline, ppl, which bring shared libraries.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​03f17c9

build/pkgs/polymake/spkg-configure.m4: In AC_RUN_IFELSE for libpolymake, use shell_enable, set_application

[mkoeppe]

Replying to dimpase:

We should check that polymake is working, not only polymake-config. (As some Perl deps might be broken [...])

I have instead added some initialization steps to the existing run test of the polymake library. Please try if it correctly rejects your broken installation

[mkoeppe]

Replying to mkoeppe:

You can add SAGE_SPKG_DEPCHECK for bliss, cddlib, lrslib, readline, ppl, which bring shared libraries.

However, this would reject polymake on homebrew, which does not provide all of these libraries. So let's not do this -- unless someone is actually running into shared library conflicts.

[mkoeppe]

Testing on arch using EXTRA_CONFIGURE_ARGS="--disable-notebook" tox -e docker-archlinux-latest-maximal.

There seems to be a packaging bug that breaks polymake when TERM is set to dumb (as we do in the (deprecated) polymake pexpect interface):

[root@ab9af06251dd sage]# TERM=dumb polymake
Welcome to polymake version 4.5
Copyright (c) 1997-2021
Ewgenij Gawrilow, Michael Joswig, and the polymake team
Technische Universität Berlin, Germany
https://polymake.org

This is free software licensed under GPL; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Loading applications now...Can't locate object method "new" via package "Term::ReadLine::Gnu" at /usr/share/polymake/perllib/Polymake/Core/Shell.pm line 30.

[dimpase]

Replying to mkoeppe:

Replying to dimpase:

We should check that polymake is working, not only polymake-config. (As some Perl deps might be broken [...])

I have instead added some initialization steps to the existing run test of the polymake library. Please try if it correctly rejects your broken installation

it does, I get the same error as from polymake in terminal. I get

configure:39826: ./conftest
terminate called after throwing an instance of 'pm::perl::exception'
  what():  "/usr/share/polymake/apps/common/perllib/PolyDB.pm", line 23: Can't locate MongoDB.pm in @INC (you may need to install the MongoDB module)
...

However, I don't get why you need this extra hack, rather than the result of perl_mongodb/spkg-configure.m4, which could be added into SAGE_SPKG_DEPCHECK, and which duly reports

perl_mongodb-none:                           no suitable system package; optional

These perl module packages aren't so dummy after all.

[dimpase]

Replying to mkoeppe:

Replying to dimpase:

Shouldn't perl_cpan_polymake_prereq (and other deps?) be in SAGE_SPKG_DEPCHECK for polymake?

perl_cpan_polymake_prereq does not make sense in SAGE_SPKG_DEPCHECK because it is a dummy script package -- we provide no spkg-install for it.

What do you mean it's dummy - it does the check for the needed modules! See also comment:98.

[mkoeppe]

"Dummy" means we have no spkg-install script; the package only exists for running the spkg-configure.m4.

[mkoeppe]

The perl_mongodb package is not required. Only users who want to use PolyDB need it. That's why I split it out from perl_cpan_polymake_prereq in #31840.

[dimpase]

Replying to mkoeppe:

"Dummy" means we have no spkg-install script; the package only exists for running the spkg-configure.m4.

anyhow, these packages do useful tests on suitability of system polymake, as we have discovered here, why not add them to SAGE_SPKG_DEPCHECK ?

[mkoeppe]

You are confused about SAGE_SPKG_DEPCHECK. This macro implements the logic "dependency library B is installed from SPKG ==> must install package A from SPKG too".

[dimpase]

Replying to mkoeppe:

The perl_mongodb package is not required. Only users who want to use PolyDB need it. That's why I split it out from perl_cpan_polymake_prereq in #31840.

I don't know what PolyDB is, it seems to be an integral part of polymake now. (At least on Gentoo there seems to be no option to have polymake not dependent on it)

[dimpase]

Replying to mkoeppe:

You are confused about SAGE_SPKG_DEPCHECK. This macro implements the logic "dependency library B is installed from SPKG ==> must install package A from SPKG too".

Note that X->Y <=> ¬Y->¬X (i.e. A from system needs B from system)

[mkoeppe]

As I said, you are confused.

[dimpase]

No, I wrote the macro SAGE_SPKG_CONFIGURE, in fact, and it's you who are confused. :-) If I do

build/pkgs/polymake/spkg-configure.m4

@@ -1 +1 @@
 SAGE_SPKG_CONFIGURE([polymake], [
     dnl Need polymake >= 3.5 for "improve callable library compatibility with threads"
     m4_pushdef([POLYMAKE_VERSION_MIN], [3.5])
+SAGE_SPKG_DEPCHECK([perl_mongodb], [
     AC_CACHE_CHECK([for polymake-config >= ]POLYMAKE_VERSION_MIN, [ac_cv_path_POLYMAKE_CONFIG], [
         AC_PATH_PROGS_FEATURE_CHECK([POLYMAKE_CONFIG], [polymake-config], [
             polymake_config_version=$($ac_path_POLYMAKE_CONFIG --version 2>&1)
@@ -41 +42 @@
            LDFLAGS="$saved_LDFLAGS"
            AC_LANG_POP([C++])
           ])
+])
     m4_popdef([POLYMAKE_VERSION_MIN])
 ])

then I get

## --------------------------------------------------------- ##
## Checking whether SageMath should install SPKG polymake... ##
## --------------------------------------------------------- ##
configure:39701: checking whether any of perl_mongodb is installed as or will be installed as SPKG
configure:39706: result: yes; install polymake as well
configure:39897: no suitable system package found for SPKG polymake

[dimpase]

Yes, I agree that checking whether any of perl_mongodb is installed as or will be installed as SPKG message is misleading, but, well...

[mkoeppe]

But it's late night in Europe, and your change there implements the wrong logic.

[dimpase]

Replying to mkoeppe:

But it's late night in Europe, and your change there implements the wrong logic.

it's not yet 1am here, and it's only wrong if perl_mongodb is optional for the latest polymake. Note that Gentoo builds polymake just fine, while there is no Perl MongoDB installed (and the result of the build is broken).

[mkoeppe]

1. perl_mongodb *is* optional for polymake.

2. Whether system perl_mongodb is available or not, we can use system polymake.

3. Even if we add an installation script for perl_mongodb, installing it as an SPKG does NOT make it necessary to stop using the system polymake (which is what SAGE_SPKG_DEPCHECK implements).

[mkoeppe]

Replying to dimpase:

Replying to mkoeppe:

Replying to dimpase:

We should check that polymake is working, not only polymake-config. (As some Perl deps might be broken [...])

I have instead added some initialization steps to the existing run test of the polymake library. Please try if it correctly rejects your broken installation

it does, I get the same error as from polymake in terminal.

Good, then spkg-configure.m4 (= the scope of this ticket) does its job correctly.

However, I don't get why you need this extra hack

Not a hack, just a proper runtest for the polymake library.

[gh-sheerluck]

I don't know how Loading applications now... works but I get

Application polytope currently uses following contributed and third-party software packages:
4ti2, bliss, cdd, flint, graphviz, libnormaliz, lrs, permlib, povray, ppl, qhull, singular, sketch, sympol, threejs, tikz, tosimplex
For more details:  show_credits;
polytope > 

so I guess I never touch PolyDB.pm somehow (I use dev-perl/MongoDB for our docker images only)

[gh-sheerluck]

I hope this ticket stays not about perl_mongodb in any way. Somehow it wasn't about perl_mongodb for Arch Linux users.

[dimpase]

Replying to mkoeppe:

1. perl_mongodb *is* optional for polymake.

2. Whether system perl_mongodb is available or not, we can use system polymake.

except that we can't in some cases, see above.

3. Even if we add an installation script for perl_mongodb, installing it as an SPKG does NOT make it necessary to stop using the system polymake (which is what SAGE_SPKG_DEPCHECK implements).

Hopefully this is not happening on my watch :-)

---

Anyhow it's good enough now.

[mkoeppe]

Thanks.