Opened 2 years ago

Last modified 3 weeks ago

#27763 needs_work enhancement

Perl packages for polymake

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-9.4
Component: packages: optional Keywords:
Cc: dimpase, jipilab, lorenz@…, kastner@…, paffenholz@… Merged in:
Authors: Matthias Koeppe Reviewers:
Report Upstream: N/A Work issues:
Branch: u/mkoeppe/perl_packages_for_polymake (Commits, GitHub, GitLab) Commit: 225dea925c37e385e74b8403d4ee294a3ca4e689
Dependencies: #29582 Stopgaps:

Status badges

Description (last modified by jipilab)

To simplify installation of polymake, we add some required CPAN packages.

Including recursive dependencies, these are 103 packages, a total of 7.5 MB of source archives, which we pack up as the source tar ball of the new spkg perl_cpan_polymake_prereq. The source tar ball (attached) also contains a copy of the self-contained cpanm Perl script, which is used for installing the packages. If SAGE_CHECK=no, only about 35 packages are installed. The rest of the dependencies are for testing.

Using an spkg-configure.m4, it is downloaded and installed only if the Perl packages are not already present on the system.

The spkg-install installs the packages via the package dependencies mechanism. If some of the packages are already installed in some version on the system, we do not install a newer version.

To try out:

   wget -P upstream https://trac.sagemath.org/raw-attachment/ticket/27763/perl_cpan_polymake_prereq-2019-05-19.tar.gz

See also the sage-devel post about compilation: https://groups.google.com/forum/#!topic/sage-devel/ukJgMweHszU

Attachments (10)

perl_cpan_polymake_prereq-2019-05-13.tar.gz (5.2 MB) - added by mkoeppe 2 years ago.
polymake-3.4.log (222.4 KB) - added by jipilab 2 years ago.
Curious install log -> perl_term_readline_gnu
perl_cpan_polymake_prereq-2019-05-13.log (33.7 KB) - added by gh-sophiasage 2 years ago.
failed install of cpan perl packages on mac
perl_packages.log (16.4 KB) - added by gh-sophiasage 2 years ago.
perl_package_debian.log (18.0 KB) - added by jipilab 2 years ago.
Failure on debian stretch
build.log (153.8 KB) - added by gh-sophiasage 2 years ago.
build.log file from failed installation of cpan perl packages on mac
perl_cpan_polymake_prereq-2019-05-17.tar.gz (6.9 MB) - added by mkoeppe 2 years ago.
perl_cpan_polymake_prereq-2019-05-19.tar.gz (7.2 MB) - added by mkoeppe 2 years ago.
polymake-3.4.2.log (130.1 KB) - added by gh-sophiasage 2 years ago.
failed installation of polymake on mac
polymake-3.4.3.log (4.5 KB) - added by jipilab 15 months ago.
polymake installation failing on debian-buster

Change History (108)

comment:1 Changed 2 years ago by mkoeppe

  • Cc dimpase added

comment:2 Changed 2 years ago by dimpase

I'd suggest to have appropriate spkg-configure.m4, to avoid installing extra...

By the way, could you provide spkg-configure.m4 for perl_term_readline_gnu, and other already present as packages polymake deps (perl or not)?

comment:3 Changed 2 years ago by mkoeppe

Do you have a suggestion how to install these CPAN packages if they are not available? There seems to be a large number of dependencies that each of the listed packages pulls in -- but probably many are "standard" on any system that has perl installed.

comment:4 Changed 2 years ago by dimpase

I don't know much about perl. Does it have anything analogous to Python's pip, which allows direct installation from the net (from CPAN, I guess) ?

comment:5 Changed 2 years ago by mkoeppe

Yes, the cpan program does the equivalent of pip.

comment:6 Changed 2 years ago by dimpase

I tried something like perl -MCPAN -e 'install XML::Writer' but it seems to want to install stuff into /usr/local rather than $SAGE_LOCAL. Do you know how to set all this up properly?

comment:7 Changed 2 years ago by mkoeppe

I have only used cpan -i, which asks whether to install system wide or in the user's home directory. Here's some info: https://stackoverflow.com/questions/540640/how-can-i-install-a-cpan-module-into-a-local-directory

I used something like this for perl_term_readline_gnu, but this is not going through cpan.

comment:8 Changed 2 years ago by dimpase

I don't know, it seems one needs to setup some local::lib magic, otherwise I keep getting errors trying to to install into /usr/local.

local::lib is a Perl module that apparently allows one to use non-standard directories for module installation, it might not be in by default (e.g. it's not in Gentoo, one needs to install it).

Anyhow - for checking in autoconf which perl modules are installed, one can use AX_PROG_PERL_MODULES

Last edited 2 years ago by dimpase (previous) (diff)

comment:9 follow-up: Changed 2 years ago by dimpase

I also had to install File::Slurp perl module. Fortunately on Gentoo there seems to be an (almost?) 1-to-1 correspondence between perl packages named X::Y and Gentoo packages named X-Y.

I also see

File "src/sage/interfaces/polymake.py", line 884, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c                                 # optional - polymake
Expected:
    cube of dimension 15
Got:
    1
    print ref(@SAGE590);
    print reftype(@SAGE590);
**********************************************************************
File "src/sage/interfaces/polymake.py", line 886, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c.N_VERTICES                      # optional - polymake
Expected:
    32768
Got:
    Member function 'N_VERTICES' of  object

comment:10 in reply to: ↑ 9 Changed 2 years ago by mkoeppe

Replying to dimpase:

I also had to install File::Slurp perl module.

Yes, this is already mentioned in polymake/SPKG.txt

Fortunately on Gentoo there seems to be an (almost?) 1-to-1 correspondence between perl packages named X::Y and Gentoo packages named X-Y.

I also see

File "src/sage/interfaces/polymake.py", line 884, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c                                 # optional - polymake
Expected:
    cube of dimension 15
Got:
    1
    print ref(@SAGE590);
    print reftype(@SAGE590);
**********************************************************************
File "src/sage/interfaces/polymake.py", line 886, in sage.interfaces.polymake.Polymake._eval_line
Failed example:
    c.N_VERTICES                      # optional - polymake
Expected:
    32768
Got:
    Member function 'N_VERTICES' of  object

With which version of polymake is this? The 3.4 upgrade ticket has fixes for something like this.

comment:11 Changed 2 years ago by dimpase

If I do

--- a/src/sage/interfaces/polymake.py
+++ b/src/sage/interfaces/polymake.py
@@ -874,10 +874,6 @@ class Polymake(ExtraTabCompletion, Expect):
         sometimes it hangs, and therefore we remove it from the tests, for now::
 
             sage: c = polymake.cube(15)             # optional - polymake
-            sage: polymake.eval('print {}->F_VECTOR;'.format(c.name()), timeout=1) # optional - polymake # not tested
-            Traceback (most recent call last):
-            ...
-            RuntimeError: Polymake fails to respond timely
 
         We verify that after the timeout, polymake is still able to give answers::
 

then the tests pass, so the timeout thing seems to be doing more harm than good.

comment:12 Changed 2 years ago by dimpase

I am testing this branch, so 3.4.

comment:13 Changed 2 years ago by dimpase

oops, sorry, wrong ticket. This is meant to be about polymake update to 3.4

comment:14 Changed 2 years ago by mkoeppe

yes, #24905.

comment:15 Changed 2 years ago by dimpase

Please see #27795 for using system's perm package Term::ReadLine::Gnu

comment:16 Changed 2 years ago by mkoeppe

  • Dependencies set to #24905, #27795

comment:17 follow-up: Changed 2 years ago by dimpase

How about adding checks for all the needed perl packages in the style of #27795 to polymake itself? Actually, it would be easier (per package), as Term::ReadLine::Gnu is non-standard in the sense one cannot just use it.

As usual, they pay a usability price by not using autotools :-)

comment:18 in reply to: ↑ 17 Changed 2 years ago by mkoeppe

  • Cc lorenz@… kastner@… paffenholz@… added

Replying to dimpase:

How about adding checks for all the needed perl packages in the style of #27795 to polymake itself?

polymake itself does check for the packages needed at build time in its configure script. The MongoDB package is checked at run time of polymake for the optional polyDB feature.

comment:19 Changed 2 years ago by mkoeppe

The challenge in this ticket is to make the installation of missing Perl packages not depend on internet access. Here's a possible approach.

First install App::cpanminus. Then

cpanm --local-lib-contained $PERL_LOCAL_LIB_ROOT --exclude-vendor --save-dists $PARTIAL_CPAN_MIRROR XML::Writer XML::LibXML XML::LibXSLT File::Slurp Term::ReadLine::Gnu JSON SVG MongoDB

will (1) download and install these prerequisites in $PERL_LOCAL_LIB_ROOT, (2) save the source packages in a directory tree $PARTIAL_CPAN_MIRROR.

These are 58 packages, a total of 5 MB of source archives, which we can pack up as the source tar ball of the new spkg perl_cpan_polymake_prereq.

Then figure out how to install with plain cpan (not cpanm, which is not available out of the box on macOS) using this local partial CPAN mirror.

comment:20 Changed 2 years ago by mkoeppe

  • Branch set to u/mkoeppe/perl_packages_for_polymake

comment:21 Changed 2 years ago by git

  • Commit set to 31b52237791b68a1d5e7231d0b1a66bffc75fb73

Branch pushed to git repo; I updated commit sha1. New commits:

31b5223Add perl_cpan_polymake_prereq as a polymake dependency

comment:22 Changed 2 years ago by git

  • Commit changed from 31b52237791b68a1d5e7231d0b1a66bffc75fb73 to 749c447ad7a5ebc266b9d27a8ff59014b771b0b8

Branch pushed to git repo; I updated commit sha1. New commits:

476c148Install CPAN modules with cpanm
7611f43Use cpanm --skip-satisfied
749c447List dependencies in Makefile.PL

comment:23 Changed 2 years ago by mkoeppe

  • Authors set to Matthias Koeppe
  • Dependencies #24905, #27795 deleted
  • Description modified (diff)
  • Status changed from new to needs_review

comment:24 Changed 2 years ago by git

  • Commit changed from 749c447ad7a5ebc266b9d27a8ff59014b771b0b8 to aecb94acc7eaeab284616da3eb4aa43b63041091

Branch pushed to git repo; I updated commit sha1. New commits:

aecb94aperl_cpan_polymake_prereq/spkg-src: Actually compress the tarball

Changed 2 years ago by mkoeppe

comment:25 Changed 2 years ago by mkoeppe

  • Description modified (diff)

comment:26 Changed 2 years ago by dimpase

  • Cc jipilab added

Changed 2 years ago by jipilab

Curious install log -> perl_term_readline_gnu

comment:27 Changed 2 years ago by jipilab

After I did install successfully polymake, the interface was broken (compile error of some sort). And tab completion did not work. Then I saw that perm_term_readline_gnu experimental package was not installed (although it seems like the configure step of the installation detected the local package but seemed like it can not use the local package).

Does that make sense? Unfortunately, I can not offer more traceback, I reached the end of my terminal buffer.

If needed, I could try to proceed similarly on a twin computer to see if I can reproduce this.

comment:28 Changed 2 years ago by dimpase

By local package you mean system package, right?

Can you uninstall Sage's perm_term_readline_gnu and then in terminal run

perl -e "use Term::ReadLine; Term::ReadLine->get_all_function_names" 
  • there should be no output if perm_term_readline_gnu is installed, and an error message otherwise.

comment:29 follow-up: Changed 2 years ago by dimpase

Also, run ./configure and see whether you have

    perl_term_readline_gnu-1.35 will not be installed (configure check)

in the output

comment:30 Changed 2 years ago by mkoeppe

I think it's possible that the system's Term::ReadLine::Gnu becomes unusable in the sage environment if sage installs its own libreadline. If for some reason we install our own libreadline, then we should probably also install Term::ReadLine::Gnu.

Last edited 2 years ago by mkoeppe (previous) (diff)

comment:31 Changed 2 years ago by mkoeppe

Or we could just revert #27795.

comment:32 follow-up: Changed 2 years ago by dimpase

when I wrote #27795 I was under impression that perl is isolated from sage's "vendored madness" libraries, as it is an independent executable, just like, say, bash.

if the problem is indeed as you described then the right way forward is to get #27277 reviewed and merged.then readline wont get built if it is available.

comment:33 Changed 2 years ago by git

  • Commit changed from aecb94acc7eaeab284616da3eb4aa43b63041091 to 10189db608570332744cdc2c51ba6c471af1d00b

Branch pushed to git repo; I updated commit sha1. New commits:

5916141Fix polymake lrslib detection
857af80polymake: add dependency ppl
10189dbMerge branch 't/27803/public/27803-polymake-lrslib-detection' into t/27763/perl_packages_for_polymake

comment:34 Changed 2 years ago by mkoeppe

  • Dependencies set to #27803

Adding #27803 dependency to prevent merge conflict.

comment:35 in reply to: ↑ 32 ; follow-up: Changed 2 years ago by mkoeppe

Replying to dimpase:

when I wrote #27795 I was under impression that perl is isolated from sage's "vendored madness" libraries, as it is an independent executable, just like, say, bash.

Yes, the Perl executable is fine, but the myriad of CPAN packages link to lots of libraries.

if the problem is indeed as you described then the right way forward is to get #27277 reviewed and merged.then readline wont get built if it is available.

OK.

comment:36 in reply to: ↑ 35 Changed 2 years ago by dimpase

Replying to mkoeppe:

Replying to dimpase:

when I wrote #27795 I was under impression that perl is isolated from sage's "vendored madness" libraries, as it is an independent executable, just like, say, bash.

Yes, the Perl executable is fine, but the myriad of CPAN packages link to lots of libraries.

yes, I understand this, but I imagine the only conflicts of this sort are possible if you run polymake as a python extension, that loads a wrong dll at runtime.

As I don't know how the polymake interface works, I can't tell whether this is the case.

In any case, could you explain where these *.so parts of perl packages may be found, so that I can look at them with ldd/otool ?

comment:37 Changed 2 years ago by dimpase

If there is indeed a possible problem like this, build/pkgs/perl_term_readline_gnu/spkg-configure.m4 needs to be modified to include a check for readline.

I would actually be more worried about a potential conflict between an incompatible readline library, say, on OSX, and the one built by Sage (or provided by Homebrew/MacPorts?)...

comment:38 in reply to: ↑ 29 ; follow-up: Changed 2 years ago by jipilab

Replying to dimpase:

Also, run ./configure and see whether you have

    perl_term_readline_gnu-1.35 will not be installed (configure check)

in the output

Yes, I met system package.

Ok, so here is the outcome.

I ran the command

perl -e "use Term::ReadLine; Term::ReadLine->get_all_function_names"

before uninstalling perl_term_readline_gnu: No output.

After uninstalling (inside of sage shell using the sage-spkg-uninstall command) I still get no output.

Things now seems to work flawlessly using the interface though.

Running ./configure, I do check the (configure check).

Ok, so now it seems to catch the system Term ReadLine. I believe I know how it happened: I did install the perl libraries using cpan -i XML::Writer XML::LibXML XML::LibXSLT File::Slurp JSON SVG as the sage -info polymake suggested, but then, the installation of the polymake package was not preceeded by a rerun of ./configure to catch the libraries that are now meanwhile to be found in my PATH? Somehow Sage still relied on its own perl_term_readline_gnu although the system one was found so it did not install the dependency. Weird.

If I run it on a clean computer, without the spkg perl_term_readline_gnu (and without any installation of the required perl libraries) I do get the error message (that now looks quite familiar to the one I was getting in Sage originally).

Ok, so in the end, I do not think there is an issue: I'll test it on fresh untouched debian machines to have a clean feedback.

Last edited 2 years ago by jipilab (previous) (diff)

comment:39 follow-up: Changed 2 years ago by dimpase

Isn't the presence of local::lib module a pre-req for all these package installations to function?

comment:40 in reply to: ↑ 38 ; follow-up: Changed 2 years ago by dimpase

Replying to jipilab:

Replying to dimpase:

Also, run ./configure and see whether you have

    perl_term_readline_gnu-1.35 will not be installed (configure check)

in the output

Yes, I met system package.

Ok, so here is the outcome.

I ran the command

perl -e "use Term::ReadLine; Term::ReadLine->get_all_function_names"

before uninstalling perl_term_readline_gnu: No output.

After uninstalling (inside of sage shell using the sage-spkg-uninstall command) I still get no output.

Things now seems to work flawlessly using the interface though.

Running ./configure, I do check the (configure check).

Ok, so now it seems to catch the system Term ReadLine. I believe I know how it happened: I did install the perl libraries using cpan -i XML::Writer XML::LibXML XML::LibXSLT File::Slurp JSON SVG as the sage -info polymake suggested, but then, the installation of the polymake package was not preceeded by a rerun of ./configure to catch the libraries that are now meanwhile to be found in my PATH? Somehow Sage still relied on its own perl_term_readline_gnu although the system one was found so it did not install the dependency. Weird.

well, this is by (somewhat unfortunate) design. Once you updated ./configure by installing/updating a spkg-configure.m4 file in build/pkgs/foo/ and you have foo installed in Sage and as a system package, the only way to switch from Sage-installed to system foo is to manually uninstall Sage's foo, and then re-run configure (othewise configure would check, and succeed in this, for Sage's foo first).

The main reason for that that some people do not want to retire the old sage -f foo as a way to install/update a package, see e.g. #27373...

comment:41 in reply to: ↑ 40 Changed 2 years ago by jipilab

well, this is by (somewhat unfortunate) design. Once you updated ./configure by installing/updating a spkg-configure.m4 file in build/pkgs/foo/ and you have foo installed in Sage and as a system package, the only way to switch from Sage-installed to system foo is to manually uninstall Sage's foo, and then re-run configure (othewise configure would check, and succeed in this, for Sage's foo first).

The main reason for that that some people do not want to retire the old sage -f foo as a way to install/update a package, see e.g. #27373...

Oh I see now. Got it! Good to know.

comment:42 in reply to: ↑ 39 Changed 2 years ago by mkoeppe

Replying to dimpase:

Isn't the presence of local::lib module a pre-req for all these package installations to function?

For installing the modules, yes. But the cpanm script is fatpacked with all its prerequisite libraries, including local::lib.

For using the modules, no. We just modify the environment variable PERL5LIB.

comment:43 follow-up: Changed 2 years ago by dimpase

surely, local::lib is an installation tool, but can you install it into system wide perl without root access?

comment:44 in reply to: ↑ 43 Changed 2 years ago by mkoeppe

Replying to dimpase:

surely, local::lib is an installation tool, but can you install it into system wide perl without root access?

A user would install local::lib into their local library by "bootstrapping" https://metacpan.org/pod/local::lib#The-bootstrapping-technique

But this is not relevant for us.

comment:45 follow-up: Changed 2 years ago by dimpase

I don't see how the local::lib thing is not relevant - one either has to say that it is a pre-req, or do that bootstrapping here.

Changed 2 years ago by gh-sophiasage

failed install of cpan perl packages on mac

comment:46 follow-up: Changed 2 years ago by gh-sophiasage

I have attached a log of the failed installation of the cpan perl packages after doing the following steps:

  • pulled the branch on this ticket
  • downloaded the tarball from this ticket and placed it in upstream folder
  • typed make build
  • typed ./sage -i polymake

Should I have followed a different procedure to test the installation on mac os mojave?

comment:47 in reply to: ↑ 45 Changed 2 years ago by mkoeppe

Replying to dimpase:

I don't see how the local::lib thing is not relevant - one either has to say that it is a pre-req, or do that bootstrapping here.

It's already solved on this ticket by shipping the self contained cpanm script. It is not needed as a prerequisite.

Changed 2 years ago by gh-sophiasage

comment:48 Changed 2 years ago by gh-sophiasage

The file perl_packages.log describes a similar failure on debian stretch when getting to MongoDB.

Changed 2 years ago by jipilab

Failure on debian stretch

comment:49 follow-up: Changed 2 years ago by jipilab

It seems that there are still some dependencies missing.

comment:50 in reply to: ↑ 46 Changed 2 years ago by mkoeppe

Replying to gh-sophiasage:

I have attached a log of the failed installation of the cpan perl packages after doing the following steps:

  • pulled the branch on this ticket
  • downloaded the tarball from this ticket and placed it in upstream folder
  • typed make build
  • typed ./sage -i polymake

Should I have followed a different procedure to test the installation on mac os mojave?

Thanks for testing! This is the correct procedure.

The relevant failure from your test on Mac is this one:

225	Building and testing Type-Tiny-1.004004 ... ! Installing Types::Standard failed. See /Users/sophiaelia/.cpanm/work/1558026103.94179/build.log for details. Retry with --force to force install it.
226	FAIL

I have seen something similar on Mac as well. Could you send the build log mentioned above?

comment:51 in reply to: ↑ 49 Changed 2 years ago by mkoeppe

Replying to jipilab:

It seems that there are still some dependencies missing.

The relevant failure on the Debian installs is this one:

40	Configuring MongoDB-v2.0.3 ... ! Finding BSON::XS (0.006000) on mirror file:///home/jplabbe/sage/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-13/src/sagepan failed.
41	! Couldn't find module or a distribution BSON::XS (0.006000)

Changed 2 years ago by gh-sophiasage

build.log file from failed installation of cpan perl packages on mac

comment:52 Changed 2 years ago by mkoeppe

Thank you! The problem on macOS seems to come from a well known problem with Module::Runtime. https://www.perlmonks.org/?node_id=1083777

comment:53 Changed 2 years ago by git

  • Commit changed from 10189db608570332744cdc2c51ba6c471af1d00b to d34f2a22d782d8110e5207c6a2d7134ea15243f5

Branch pushed to git repo; I updated commit sha1. New commits:

e6a8da0perl_cpan_polymake_prereq: Make spkg-src more robust
00a09d5Regenerate src package with new script
d34f2a2perl_cpan_polymake_prereq: Install prereqs in two stages, fixup LD environment variable for Moose

comment:54 Changed 2 years ago by mkoeppe

I have made a more robust version of the spkg-src script, which (hopefully!) makes sure to pack up all dependencies (instead of picking some up from the local installation) and also addresses the Module::Runtime version problem on macOS.

To test, first get rid of the Perl library installation in $SAGE_LOCAL and then reinstall the packages:

  rm -rf local/lib/perl5
  ./sage -f  perl_cpan_polymake_prereq perl_term_readline_gnu

Changed 2 years ago by mkoeppe

comment:55 Changed 2 years ago by mkoeppe

  • Description modified (diff)

comment:56 Changed 2 years ago by git

  • Commit changed from d34f2a22d782d8110e5207c6a2d7134ea15243f5 to 18d109a05a23e923867fba9e81e7bef46d689701

Branch pushed to git repo; I updated commit sha1. New commits:

18d109aAdd build/pkgs/perl_cpan_polymake_prereq/stage1/Makefile.PL

comment:57 Changed 2 years ago by git

  • Commit changed from 18d109a05a23e923867fba9e81e7bef46d689701 to 06fdaf2f2cd23407f1f67128000444b492b1c8c2

Branch pushed to git repo; I updated commit sha1. New commits:

06fdaf2perl_cpan_polymake_prereq: Clean up spkg-src, spkg-install

comment:58 Changed 2 years ago by git

  • Commit changed from 06fdaf2f2cd23407f1f67128000444b492b1c8c2 to 4363e80e2c19b227109e70a1e3e901b9f62b3d5b

Branch pushed to git repo; I updated commit sha1. New commits:

4363e80Add missing cpan packages

Changed 2 years ago by mkoeppe

comment:59 Changed 2 years ago by git

  • Commit changed from 4363e80e2c19b227109e70a1e3e901b9f62b3d5b to 188fcb70d5b256421dcb4cefec937982e6cb59e5

Branch pushed to git repo; I updated commit sha1. New commits:

188fcb7Update polymake SPKG.txt

comment:60 Changed 2 years ago by mkoeppe

  • Description modified (diff)

comment:61 Changed 2 years ago by git

  • Commit changed from 188fcb70d5b256421dcb4cefec937982e6cb59e5 to 3f5e3a2ce1df175860b708632c9fbccf45df0f00

Branch pushed to git repo; I updated commit sha1. New commits:

3f5e3a2Disable perl package tests unless SAGE_CHECK=yes

comment:62 Changed 2 years ago by mkoeppe

  • Description modified (diff)

Ready for another round of testing. Works for me on macOS Mojave and on a CentOS Linux release 7.6.1810 with almost no Perl stuff installed.

Changed 2 years ago by gh-sophiasage

failed installation of polymake on mac

comment:63 follow-up: Changed 2 years ago by gh-sophiasage

I have attached a log of a failed installation of polymake on macOS Mojave after pulling this ticket and doing the following steps.

  • pulled the branch on this ticket
  • downloaded the tarball perl_cpan_polymake_prereq-2019-05-19.tar.gz and put it in the upstream folder
  • in upstream folder typed rm perl_cpan_polymake_prereq_2019_05_13.tar.gz
  • in my sage directory typed make build
  • typed ./sage -f perl_term_readline_gnu
  • typed ./sage -i polymake

perl_cpan_polymake_prereq-2019-05-19 installed successfully.

Last edited 2 years ago by gh-sophiasage (previous) (diff)

comment:64 in reply to: ↑ 63 ; follow-up: Changed 2 years ago by dimpase

Replying to gh-sophiasage:

I have attached a log of a failed installation of polymake on macOS Mojave after pulling this ticket and doing the following steps.

  • pulled the branch on this ticket
  • downloaded the tarball perl_cpan_polymake_prereq-2019-05-19.tar.gz and put it in the upstream folder
  • in upstream folder typed rm perl_cpan_polymake_prereq_2019_05_13.tar.gz

why would you download the file, and then delete it? A typo?

  • in my sage directory typed make build
  • typed ./sage -f perl_term_readline_gnu
  • typed ./sage -i polymake

perl_cpan_polymake_prereq-2019-05-19 installed successfully.

I wonder why you have C++14 in your CXXFLAGS. Do you by any chance have CXXFLAGS environment variable set to something non-empty?

comment:65 follow-up: Changed 2 years ago by mkoeppe

Try deleting the installed polymake headers in local/include first. Looks like they are taking precedence in the compilation - this looks like a bug in the polymake build scripts

comment:66 in reply to: ↑ 64 Changed 2 years ago by mkoeppe

Replying to dimpase:

Replying to gh-sophiasage:

I have attached a log of a failed installation of polymake on macOS Mojave after pulling this ticket and doing the following steps.

  • pulled the branch on this ticket
  • downloaded the tarball perl_cpan_polymake_prereq-2019-05-19.tar.gz and put it in the upstream folder
  • in upstream folder typed rm perl_cpan_polymake_prereq_2019_05_13.tar.gz

why would you download the file, and then delete it? A typo?

Old version deleted

comment:67 in reply to: ↑ 65 Changed 2 years ago by gh-sophiasage

Replying to mkoeppe:

Try deleting the installed polymake headers in local/include first. Looks like they are taking precedence in the compilation - this looks like a bug in the polymake build scripts

It worked!!

comment:68 Changed 2 years ago by mkoeppe

Thanks for testing! I have updated the install scripts to take care of uninstallation in #27855.

comment:69 Changed 2 years ago by mkoeppe

  • Description modified (diff)

comment:70 follow-up: Changed 23 months ago by mkoeppe

Any takers for the review of this ticket?

comment:71 Changed 23 months ago by dimpase

I am on vacation until 4 June, can’t possibly do it before.

comment:72 Changed 23 months ago by mkoeppe

Perhaps Term::ReadKey should be added as well.

comment:73 Changed 22 months ago by embray

  • Milestone changed from sage-8.8 to sage-8.9

Moving tickets from the Sage 8.8 milestone that have been actively worked on in the last six months to the next release milestone (optimistically).

comment:74 in reply to: ↑ 70 Changed 20 months ago by jipilab

Replying to mkoeppe:

Any takers for the review of this ticket?

We'll resume our Friday morning's sage session soon. We'll be able to test it on a few virgin machines to test the robustness of installation...

comment:75 Changed 18 months ago by jipilab

  • Description modified (diff)

comment:76 follow-up: Changed 18 months ago by jhpalmieri

First, I'm getting a merge conflict with the latest version of Sage.

Second, the build is failing for me with OS X 10.14.6. The log file (with some boring lines snipped):

Found local metadata for perl_cpan_polymake_prereq-2019-05-19
Using cached file /Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/upstream/perl_cpan_polymake_prereq-2019-05-19.tar.gz
perl_cpan_polymake_prereq-2019-05-19
====================================================
Setting up build directory for perl_cpan_polymake_prereq-2019-05-19
Finished extraction
No patch files found in ../patches
****************************************************
Host system:
Darwin jpalmieri138.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64
****************************************************
C compiler: gcc
C compiler version:
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 11.0.0 (clang-1100.0.33.12)
Target: x86_64-apple-darwin18.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
****************************************************
Package 'perl_cpan_polymake_prereq' is currently not installed
No legacy uninstaller found for 'perl_cpan_polymake_prereq'; nothing to do
cpanm (App::cpanminus) 1.7044 on perl 5.018004 built for darwin-thread-multi-2level
Work directory is /Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-19/src/cpanm_home/work/1572630684.99164
You have make /usr/bin/make
You have LWP 6.05
You have /usr/bin/tar: bsdtar 2.8.3 - libarchive 2.8.3
You have /usr/bin/unzip
Entering /Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-19/stage1
Running Makefile.PL
--> Working on .
Configuring /Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-19/stage1 ... Warning: prerequisite Moose 2.2 not found. We have 2.1202.
Writing Makefile for PolymakeStage1
Writing MYMETA.yml and MYMETA.json
Checking dependencies from MYMETA.json ...
Checking if you have ExtUtils::MakeMaker 0 ... Yes (6.66)
Checking if you have Moose 2.2 ... No (2.1202 < 2.2)
Searching Moose on mirror file:///Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-19/src/sagepan ...
Downloading index file file:///Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-19/src/sagepan/modules/02packages.details.txt.gz ...
Uncompressing index file...
Unpacking Moose-2.2011.tar.gz
OK
==> Found dependencies: Moose
--> Working on Moose
Fetching file:///Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-19/src/sagepan/authors/id/E/ET/ETHER/Moose-2.2011.tar.gz ... OK
x Moose-2.2011/
x Moose-2.2011/author/
...
  [snip]
...
x Moose-2.2011/author/doc-generator
x Moose-2.2011/author/extract-inline-tests
x Moose-2.2011/author/find-dupe-test-numbers
Entering Moose-2.2011
Checking configure dependencies from META.json
Checking if you have ExtUtils::MakeMaker 6.58 ... Yes (6.66)
Checking if you have Dist::CheckConflicts 0.02 ... Yes (0.10)
Running Makefile.PL
Configuring Moose-2.2011 ... HASCOMPILERJXuX/TESTGBHw.c:2:10: fatal error: 'EXTERN.h' file not found
#include "EXTERN.h"
         ^~~~~~~~~~
1 error generated.
Couldn't execute gcc  -g -pipe -fno-common -DPERL_DARWIN -fno-strict-aliasing -fstack-protector -Os "-I/System/Library/Perl/5.18/darwin-thread-multi-2level/CORE"   -c HASCOMPILERJXuX/TESTGBHw.c -o HASCOMPILERJXuX/TESTGBHw.o: Inappropriate ioctl for device at Makefile.PL line 12.
This distribution requires a working compiler at Makefile.PL line 12.
! Configure failed for Moose-2.2011. See /Users/jpalmier/Desktop/Sage/sage_builds/PYTHON3/sage-9.0.beta3/local/var/tmp/sage/build/perl_cpan_polymake_prereq-2019-05-19/src/cpanm_home/work/1572630684.99164/build.log for details.
! Installing the dependencies failed: Installed version (2.1202) of Moose is not in range '2.2'
! Bailing out the installation for PolymakeStage1-0.
N/A

real	0m0.905s
user	0m0.561s
sys	0m0.291s
************************************************************************
Error installing package perl_cpan_polymake_prereq-2019-05-19
************************************************************************

comment:77 Changed 17 months ago by git

  • Commit changed from 3f5e3a2ce1df175860b708632c9fbccf45df0f00 to cce6cbd221634fc36df0661fd69d4f7401c5d5d7

Branch pushed to git repo; I updated commit sha1. New commits:

cce6cbdMerge tag '9.0.beta6' into t/27763/perl_packages_for_polymake

comment:78 Changed 17 months ago by dimpase

  • Milestone changed from sage-8.9 to sage-9.0

comment:79 in reply to: ↑ 76 Changed 17 months ago by mkoeppe

Replying to jhpalmieri:

the build is failing for me with OS X 10.14.6.

Configuring Moose-2.2011 ... HASCOMPILERJXuX/TESTGBHw.c:2:10: fatal error: 'EXTERN.h' file not found
#include "EXTERN.h"
         ^~~~~~~~~~
1 error generated.

From sage -info polymake:

On Mac OS X, to build Term::Readline::Gnu, on macOS 10.14 (Mojave), one
needs to do the following:

 sudo installer -pkg /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg -target /

or a "fatal error: 'EXTERN.h' file not found" will be signalled.

comment:80 Changed 17 months ago by jhpalmieri

I think that you shouldn't have the build fail; you should instead test for the headers and provide a helpful error message if they're not present. I doubt that most users will know to use sage -info polymake.

Also, the error message is out-of-date: the current version of OS X is 10.15, and there is no directory Packages in /Library/Developer/CommandLineTools/.

comment:81 follow-up: Changed 17 months ago by mkoeppe

Yes, I agree, this needs a better solution.

comment:82 in reply to: ↑ 81 Changed 17 months ago by mkoeppe

Some references for the EXTERN.h problem on macOS Mojave and newer:

comment:83 Changed 16 months ago by embray

  • Milestone changed from sage-9.0 to sage-9.1

Ticket retargeted after milestone closed

comment:84 Changed 15 months ago by jipilab

After a clean installation of 9.1.beta2, running sage -i polymake finishes with success and installs both

  • perl_cpan_polymake_prereq...............2019-05-19 (2019-05-19)
  • perl_term_readline_gnu..................1.35 (1.35)

Now, trying out polymake in sage essentially does not work...

  • Tab completion on polymake. does not work (which is an improvement, before it used to kill sage)
  • If I type c = polytopes.cube(backend='polymake') it simply never finishes. Thought I can kill it succesfully with ctrl+c.

Starting sage anew and trying:

  • p = Polyhedron(vertices= [[0],[1]],backend='polymake') also simply never gives back the prompt.

Without starting sage anew, if I simply repeat the previous operation, I get directly:

Traceback (most recent call last):
TypeError: argument should be integer or bytes-like object, not 'str'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
...

/home/jplabbe/sage/local/lib/python3.7/site-packages/sage/interfaces/interface.py in __init__(self, parent, value, is_name, name)
    718                 self._name = parent._create(value, name=name)
    719             except (TypeError, RuntimeError, ValueError) as x:
--> 720                 raise TypeError(x)
    721 
    722     def _latex_(self):

TypeError: error evaluating "@SAGE1=("Rational");":
argument should be integer or bytes-like object, not 'str'

...

*That said*, I have to mention that I do not have jupymake installed which is an optional package:

jupymake................................0.9 (not_installed)

I wanted to try a simple "sage -i polymake" from a virgin install to see how it would perform...

My os is debian-buster. As far as the installation goes, it seems that polymake runs fine under sage -polymake...

comment:85 follow-up: Changed 15 months ago by jipilab

On another debian-buster I get an error in the installation. Relevant log attached.

Changed 15 months ago by jipilab

polymake installation failing on debian-buster

comment:86 in reply to: ↑ 85 Changed 15 months ago by jipilab

Replying to jipilab:

On another debian-buster I get an error in the installation. Relevant log attached.

Turns out the same failure happens on current develop. So it's not related to this ticket necessarily. (This machine is from the department, we just got a new os update, so we are less aware of the default installations)

comment:87 Changed 13 months ago by git

  • Commit changed from cce6cbd221634fc36df0661fd69d4f7401c5d5d7 to b74fc93c12be28adac7afcad571464b063b3e509

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

64fe021perl_cpan_polymake_prereq/spkg-src: Actually compress the tarball
dfa2c3dpolymake: add dependency ppl
c1c3fd3perl_cpan_polymake_prereq: Make spkg-src more robust
ddec16fRegenerate src package with new script
98450c8perl_cpan_polymake_prereq: Install prereqs in two stages, fixup LD environment variable for Moose
99c1193Add build/pkgs/perl_cpan_polymake_prereq/stage1/Makefile.PL
118ee1fperl_cpan_polymake_prereq: Clean up spkg-src, spkg-install
08fcc25Add missing cpan packages
954e55bUpdate polymake SPKG.txt
b74fc93Disable perl package tests unless SAGE_CHECK=yes

comment:88 Changed 13 months ago by mkoeppe

  • Dependencies #27803 deleted
  • Status changed from needs_review to needs_work

comment:89 Changed 13 months ago by jipilab

Let me know when this is good to be tested: turns out, I might have more time now for testing(!)...

comment:90 Changed 12 months ago by mkoeppe

  • Milestone changed from sage-9.1 to sage-9.2

pushing these forward to 9.2

comment:91 Changed 12 months ago by git

  • Commit changed from b74fc93c12be28adac7afcad571464b063b3e509 to e78723863936a485d354c53f38e1c73d57ae713f

Branch pushed to git repo; I updated commit sha1. New commits:

e787238Merge tag '9.1.rc1' into t/27763/perl_packages_for_polymake

comment:92 Changed 12 months ago by git

  • Commit changed from e78723863936a485d354c53f38e1c73d57ae713f to 20b8ffe35d1a326c0ba808cf1b150e207159d22f

Branch pushed to git repo; I updated commit sha1. New commits:

8c65337polymake SPKG.txt: Remove outdated info regarding macOS headers
20b8ffeMerge branch 't/29582/polymake_spkg_txt__remove_outdated_info_regarding_macos_headers' into t/27763/perl_packages_for_polymake

comment:93 Changed 12 months ago by mkoeppe

  • Dependencies set to #29582
Last edited 12 months ago by mkoeppe (previous) (diff)

comment:94 Changed 12 months ago by git

  • Commit changed from 20b8ffe35d1a326c0ba808cf1b150e207159d22f to 225dea925c37e385e74b8403d4ee294a3ca4e689

Branch pushed to git repo; I updated commit sha1. New commits:

225dea9Merge tag '9.1.rc2' into t/27763/perl_packages_for_polymake

comment:95 follow-up: Changed 12 months ago by mkoeppe

NOT ready for new tests.

comment:96 in reply to: ↑ 95 Changed 11 months ago by jipilab

Replying to mkoeppe:

NOT ready for new tests.

Ok, just I'll wait until you change the flag to needs review and then I could run some installation tests.

comment:97 Changed 7 months ago by mkoeppe

  • Milestone changed from sage-9.2 to sage-9.3

comment:98 Changed 3 weeks ago by mkoeppe

  • Milestone changed from sage-9.3 to sage-9.4

Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.

Note: See TracTickets for help on using tickets.