Opened 5 years ago
Last modified 4 years ago
#22984 closed enhancement
Upgrade normaliz to 3.5.1 and pynormaliz to 1.10 — at Version 21
Reported by:  Matthias Köppe  Owned by:  

Priority:  major  Milestone:  sage8.3 
Component:  packages: optional  Keywords:  IMAPolyGeom 
Cc:  Winfried Bruns, Thierry Monteil, Vincent Delecroix, Travis Scrimshaw, JeanPhilippe Labbé, Sebastian Gutsche  Merged in:  
Authors:  Matthias Koeppe  Reviewers:  Travis Scrimshaw, Thierry Monteil 
Report Upstream:  Completely fixed; Fix reported upstream  Work issues:  
Branch:  u/mkoeppe/upgrade_normaliz_to_3_4_0_and_pynormaliz_to_1_7 (Commits, GitHub, GitLab)  Commit:  b8f7abc04085155225e54ad168ce4a7884fdd6cf 
Dependencies:  Stopgaps: 
Description (last modified by )
Among many new features and improvements, these new versions make computations interruptible with ^C
.
Source tarballs:
 https://github.com/Normaliz/Normaliz/releases/download/v3.5.1/normaliz3.5.1.tar.gz
 https://pypi.python.org/packages/47/99/c9043d8392f99c9c4013eb82ce931f75ef64d6fd665cd8d008ce80be881e/PyNormaliz1.10.tar.gz#md5=89b212de8e833205b9776b1ef90c3585
Upstream bug: https://github.com/Normaliz/Normaliz/issues/176
Change History (21)
comment:1 Changed 5 years ago by
Description:  modified (diff) 

Milestone:  sage8.0 → sage8.1 
Summary:  Upgrade normaliz to 3.3.0 and pynormaliz to 1.7 → Upgrade normaliz to 3.4.0 and pynormaliz to 1.7 
comment:2 Changed 5 years ago by
Branch:  → u/mkoeppe/upgrade_normaliz_to_3_4_0_and_pynormaliz_to_1_7 

comment:3 Changed 5 years ago by
Cc:  Thierry Monteil added 

Commit:  → a6ccc4235fa65a26fd772ad963df01b9ece5633f 
Description:  modified (diff) 
Summary:  Upgrade normaliz to 3.4.0 and pynormaliz to 1.7 → Upgrade normaliz to 3.4.0 and pynormaliz to 1.7+ 
comment:4 Changed 5 years ago by
Description:  modified (diff) 

comment:5 Changed 5 years ago by
Authors:  → Matthias Koeppe 

comment:6 Changed 5 years ago by
Commit:  a6ccc4235fa65a26fd772ad963df01b9ece5633f → 5e7a45b55eb85040b4a4c84fc0c8bd6506fdbdad 

Branch pushed to git repo; I updated commit sha1. New commits:
5e7a45b  Update PyNormaliz to 1.8

comment:7 Changed 5 years ago by
Description:  modified (diff) 

Summary:  Upgrade normaliz to 3.4.0 and pynormaliz to 1.7+ → Upgrade normaliz to 3.4.0 and pynormaliz to 1.8 
comment:8 Changed 5 years ago by
Description:  modified (diff) 

Milestone:  sage8.1 → sage8.2 
Summary:  Upgrade normaliz to 3.4.0 and pynormaliz to 1.8 → Upgrade normaliz to 3.5.1 and pynormaliz to 1.10 
comment:9 Changed 5 years ago by
Commit:  5e7a45b55eb85040b4a4c84fc0c8bd6506fdbdad → f2137cc5bb49278657ce23f0f8c775400b494023 

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
f6011b4  Upgrade normaliz to 3.5.1 and pynormaliz to 1.10

5536407  normaliz: Add flint dependency

71d2194  normaliz: Back out 22684_cone_reduce_memory_usage.patch

f2137cc  Polyhedron_normaliz: Adjust doctest to 'uniqueness in output' of normaliz 3.5.1

comment:10 Changed 5 years ago by
Cc:  Vincent Delecroix Travis Scrimshaw JeanPhilippe Labbé added 

Status:  new → needs_review 
comment:11 Changed 5 years ago by
Description:  modified (diff) 

comment:12 Changed 5 years ago by
Reviewers:  → Travis Scrimshaw 

Status:  needs_review → needs_work 
I am now getting this crash (looks like an upstream problem):
sage: ieqs = [[0, 2, 0, 1, 0, 0, 0, 0, 0], [0, 0, 2, 0, 1, 0, 0, 0, 0], ....: [1, 1, 0, 2, 1, 0, 0, 0, 0], [0, 0, 1, 1, 2, 1, 0, 0, 0], ....: [0, 0, 0, 0, 1, 2, 1, 0, 0], [0, 0, 0, 0, 0, 1, 2, 1, 0], ....: [0, 0, 0, 0, 0, 0, 1, 2, 1], [2, 0, 0, 0, 0, 0, 0, 1, 2], ....: [0, 1, 0, 0, 0, 0, 0, 0, 0], [0, 0, 1, 0, 0, 0, 0, 0, 0], ....: [0, 0, 0, 1, 0, 0, 0, 0, 0], [0, 0, 0, 0, 1, 0, 0, 0, 0], ....: [0, 0, 0, 0, 0, 1, 0, 0, 0], [0, 0, 0, 0, 0, 0, 1, 0, 0], ....: [0, 0, 0, 0, 0, 0, 0, 1, 0], [0, 0, 0, 0, 0, 0, 0, 0, 1], ....: [1, 1, 1, 1, 1, 1, 1, 1, 1], [3, 1, 0, 0, 0, 0, 0, 0, 0], ....: [4, 0, 1, 0, 0, 0, 0, 0, 0], [6, 0, 0, 1, 0, 0, 0, 0, 0], ....: [8, 0, 0, 0, 1, 0, 0, 0, 0], [6, 0, 0, 0, 0, 1, 0, 0, 0], ....: [4, 0, 0, 0, 0, 0, 1, 0, 0], [2, 0, 0, 0, 0, 0, 0, 1, 0], ....: [0, 0, 0, 0, 0, 0, 0, 0, 1]] sage: P = Polyhedron(ieqs=ieqs, backend='normaliz') sage: P.integral_points() python2: ./libnormaliz/matrix.cpp:2901: libnormaliz::Matrix<Integer> libnormaliz::Matrix<Integer>::LLL_red(libnormaliz::Matrix<Integer>&, libnormaliz::Matrix<Integer>&) const [with Integer = double]: Assertion `(int) rank()==n' failed.  /home/travis/sagebuild/local/lib/python2.7/sitepackages/cysignals/signals.so(+0x574b)[0x7fa5afdf274b] /home/travis/sagebuild/local/lib/python2.7/sitepackages/cysignals/signals.so(+0x57b5)[0x7fa5afdf27b5] /home/travis/sagebuild/local/lib/python2.7/sitepackages/cysignals/signals.so(+0x849a)[0x7fa5afdf549a] /lib/x86_64linuxgnu/libpthread.so.0(+0x11390)[0x7fa5bcd37390] /lib/x86_64linuxgnu/libc.so.6(gsignal+0x38)[0x7fa5bc991428] /lib/x86_64linuxgnu/libc.so.6(abort+0x16a)[0x7fa5bc99302a] /lib/x86_64linuxgnu/libc.so.6(+0x2dbd7)[0x7fa5bc989bd7] /lib/x86_64linuxgnu/libc.so.6(+0x2dc82)[0x7fa5bc989c82] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZNK11libnormaliz6MatrixIdE7LLL_redERS1_S2_+0x5b9)[0x7fa47314f4a9] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZNK11libnormaliz6MatrixIdE17LLL_red_transposeERS1_S2_+0x83)[0x7fa47314fdc3] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZN11libnormaliz15LLL_coordinatesIxdEENS_25Sublattice_RepresentationIT_EERKNS_6MatrixIT0_EE+0x56)[0x7fa4730cfde6] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZN11libnormaliz31LLL_coordinates_without_1st_colIxxEEvRNS_25Sublattice_RepresentationIT_EENS_6MatrixIT0_EES7_b+0x291)[0x7fa4730d0881] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZN11libnormaliz14ProjectAndLiftIxxE7computeEbb+0x194)[0x7fa473104e54] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZN11libnormaliz4ConeI10__gmp_exprIA1_12__mpz_structS3_EE16project_and_liftERNS_14ConePropertiesERNS_6MatrixIS4_EERKS9_SA_b+0x839)[0x7fa473179b69] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZN11libnormaliz4ConeI10__gmp_exprIA1_12__mpz_structS3_EE31try_approximation_or_projectionERNS_14ConePropertiesE+0xf00)[0x7fa4731f5370] /home/travis/sagebuild/local/lib/libnormaliz.so.3(_ZN11libnormaliz4ConeI10__gmp_exprIA1_12__mpz_structS3_EE7computeENS_14ConePropertiesE+0x1e1)[0x7fa4731e8671] /home/travis/sagebuild/local/lib/python2.7/sitepackages/PyNormaliz_cpp.so(_Z14_NmzResultImplI10__gmp_exprIA1_12__mpz_structS2_EEP7_objectPN11libnormaliz4ConeIT_EES5_+0x75)[0x7fa4734f6cc5] /home/travis/sagebuild/local/lib/python2.7/sitepackages/PyNormaliz_cpp.so(_Z10_NmzResultP7_objectS0_+0x150)[0x7fa4734ed890]
Originally found by
sage: RC = RiggedConfigurations(['E',8,1], [[3,2]]) sage: len(RC.module_generators) python2: ./libnormaliz/matrix.cpp:2901: libnormaliz::Matrix<Integer> libnormaliz::Matrix<Integer>::LLL_red(libnormaliz::Matrix<Integer>&, libnormaliz::Matrix<Integer>&) const [with Integer = double]: Assertion `(int) rank()==n' failed.
(Mathematical note, through trying to find a smaller example, I am noticing that E_{8}^{(1)} is special in that it is the only place where I see nonfulldimensional nonempty polytopes in my computations. Well, I also see a 0dimensional polytope for type D_{4}^{(3)}, but that doesn't cause a crash as it is likely specialcased.)
Also can you update the comment before enableflint
to also say you enable flint as it is part of the standard packages of Sage?
comment:13 Changed 5 years ago by
Commit:  f2137cc5bb49278657ce23f0f8c775400b494023 → b8f7abc04085155225e54ad168ce4a7884fdd6cf 

Branch pushed to git repo; I updated commit sha1. New commits:
b8f7abc  normaliz/spkginstall: Explain enableflint

comment:14 Changed 5 years ago by
I can confirm this crash. Here on Mac OS X, this actually becomes an unhandled SIGABRT:
Assertion failed: ((int) rank()==n), function LLL_red, file ./libnormaliz/matrix.cpp, line 2901.  0 signals.so 0x000000010af572d8 print_backtrace + 40 1 ??? 0x000000010dcac1e0 0x0 + 4526359008  Unhandled SIGABRT: An abort() occurred. This probably occurred because a *compiled* module has a bug in it and is not properly wrapped with sig_on(), sig_off(). Python will now terminate.
comment:15 Changed 5 years ago by
Upstream master branch (8db5b68491a105b9185a01c29c2f126f5115e238) has different behavior but does not fix this.
comment:16 Changed 5 years ago by
Reviewers:  Travis Scrimshaw → Travis Scrimshaw, Thierry Monteil 

I confirm that the packages build and pass selftests and doctests correctly on both 64 and 32 bit architectures on Debian stretch. I also comfirm that there is a crash on both architectures regarding the test provided by Travis (which should be added in our doctests, once the issue is fixed).
comment:17 Changed 5 years ago by
I am not sure what the matrix ieqs represents. My interpretation: inhomogeneous inequalities, and we want to compute the lattice points in the polytope they define. Under this assumption I have made a Normaliz input file rash.in
amb_space auto inhom_inequalities [[0, 2, 0, 1, 0, 0, 0, 0, 0], ... [0, 0, 0, 0, 0, 0, 0, 0, 1]] HilbertBasis
Then a run of
normaliz c crash.in
finishes without any problems in 3.5.1 and the intended 3.5.2. It turns out that the polytope is empty:
0 module generators 1252 Hilbert basis elements of recession monoid 0 vertices of polyhedron 39 extreme rays of recession cone 11 support hyperplanes of polyhedron (homogenized) embedding dimension = 9 affine dimension of the polyhedron = 1 rank of recession monoid = 8 ...
Is my interpretation of the input correct?
comment:18 Changed 5 years ago by
Of course, if the matrix represents inhomogeneous inequalities, what is the RHS? My input above assumes it is the last coordinate. I will now try with the first.
comment:19 Changed 5 years ago by
Unfortunately Normaliz crashes when coordinate 1 is interpreted as the RHS. Will take care of it.
comment:20 followup: 21 Changed 5 years ago by
The problem is solved. Replace line 323 of source/libnormaliz/sublattice_representation.h
by
if(Vertices.nr_of_rows()==0  Vertices.rank()<EmbDim){ // use Supps for LLL coordinates
i.e., delete "1" in this line.
comment:21 Changed 5 years ago by
Description:  modified (diff) 

Report Upstream:  N/A → Reported upstream. Developers acknowledge bug. 
Replying to Winfried:
The problem is solved. Replace line 323 of source/libnormaliz/sublattice_representation.h
by
if(Vertices.nr_of_rows()==0  Vertices.rank()<EmbDim){ // use Supps for LLL coordinatesi.e., delete "1" in this line.
Great, thank you for figuring it out. I am guessing you are in the process of making a commit upstream or should I do a pull request?
Tested on Mac OS X. Works well except for interrupting (may require newer pynormaliz...) and the doctest failures reported in #23586 (fixing broken pynormaliz doctests).
New commits:
Upgrade normaliz, pynormaliz
normaliz: Back out 22684_cone_reduce_memory_usage.patch