Opened 5 years ago

Last modified 4 years ago

#22984 closed enhancement

Upgrade normaliz to 3.5.1 and pynormaliz to 1.10 — at Version 21

Reported by: Matthias Köppe Owned by:
Priority: major Milestone: sage-8.3
Component: packages: optional Keywords: IMA-PolyGeom
Cc: Winfried Bruns, Thierry Monteil, Vincent Delecroix, Travis Scrimshaw, Jean-Philippe Labbé, Sebastian Gutsche Merged in:
Authors: Matthias Koeppe Reviewers: Travis Scrimshaw, Thierry Monteil
Report Upstream: Completely fixed; Fix reported upstream Work issues:
Branch: u/mkoeppe/upgrade_normaliz_to_3_4_0_and_pynormaliz_to_1_7 (Commits, GitHub, GitLab) Commit: b8f7abc04085155225e54ad168ce4a7884fdd6cf
Dependencies: Stopgaps:

Status badges

Change History (21)

comment:1 Changed 5 years ago by Matthias Köppe

Description: modified (diff)
Milestone: sage-8.0sage-8.1
Summary: Upgrade normaliz to 3.3.0 and pynormaliz to 1.7Upgrade normaliz to 3.4.0 and pynormaliz to 1.7

comment:2 Changed 5 years ago by Matthias Köppe

Branch: u/mkoeppe/upgrade_normaliz_to_3_4_0_and_pynormaliz_to_1_7

comment:3 Changed 5 years ago by Matthias Köppe

Cc: Thierry Monteil added
Commit: a6ccc4235fa65a26fd772ad963df01b9ece5633f
Description: modified (diff)
Summary: Upgrade normaliz to 3.4.0 and pynormaliz to 1.7Upgrade normaliz to 3.4.0 and pynormaliz to 1.7+

Tested on Mac OS X. Works well except for interrupting (may require newer pynormaliz...) and the doctest failures reported in #23586 (fixing broken pynormaliz doctests).


New commits:

be0b9d2Upgrade normaliz, pynormaliz
a6ccc42normaliz: Back out 22684_cone_reduce_memory_usage.patch

comment:4 Changed 5 years ago by Matthias Köppe

Description: modified (diff)

comment:5 Changed 5 years ago by Matthias Köppe

Authors: Matthias Koeppe

comment:6 Changed 5 years ago by git

Commit: a6ccc4235fa65a26fd772ad963df01b9ece5633f5e7a45b55eb85040b4a4c84fc0c8bd6506fdbdad

Branch pushed to git repo; I updated commit sha1. New commits:

5e7a45bUpdate PyNormaliz to 1.8

comment:7 Changed 5 years ago by Matthias Köppe

Description: modified (diff)
Summary: Upgrade normaliz to 3.4.0 and pynormaliz to 1.7+Upgrade normaliz to 3.4.0 and pynormaliz to 1.8

comment:8 Changed 5 years ago by Matthias Köppe

Description: modified (diff)
Milestone: sage-8.1sage-8.2
Summary: Upgrade normaliz to 3.4.0 and pynormaliz to 1.8Upgrade normaliz to 3.5.1 and pynormaliz to 1.10

comment:9 Changed 5 years ago by git

Commit: 5e7a45b55eb85040b4a4c84fc0c8bd6506fdbdadf2137cc5bb49278657ce23f0f8c775400b494023

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

f6011b4Upgrade normaliz to 3.5.1 and pynormaliz to 1.10
5536407normaliz: Add flint dependency
71d2194normaliz: Back out 22684_cone_reduce_memory_usage.patch
f2137ccPolyhedron_normaliz: Adjust doctest to 'uniqueness in output' of normaliz 3.5.1

comment:10 Changed 5 years ago by Matthias Köppe

Cc: Vincent Delecroix Travis Scrimshaw Jean-Philippe Labbé added
Status: newneeds_review

comment:11 Changed 5 years ago by Matthias Köppe

Description: modified (diff)

comment:12 Changed 5 years ago by Travis Scrimshaw

Reviewers: Travis Scrimshaw
Status: needs_reviewneeds_work

I am now getting this crash (looks like an upstream problem):

sage: ieqs = [[0, 2, 0, -1, 0, 0, 0, 0, 0], [0, 0, 2, 0, -1, 0, 0, 0, 0],
....:         [1, -1, 0, 2, -1, 0, 0, 0, 0], [0, 0, -1, -1, 2, -1, 0, 0, 0],
....:         [0, 0, 0, 0, -1, 2, -1, 0, 0], [0, 0, 0, 0, 0, -1, 2, -1, 0],
....:         [0, 0, 0, 0, 0, 0, -1, 2, -1], [2, 0, 0, 0, 0, 0, 0, -1, 2],
....:         [0, -1, 0, 0, 0, 0, 0, 0, 0], [0, 0, -1, 0, 0, 0, 0, 0, 0],
....:         [0, 0, 0, -1, 0, 0, 0, 0, 0], [0, 0, 0, 0, -1, 0, 0, 0, 0],
....:         [0, 0, 0, 0, 0, -1, 0, 0, 0], [0, 0, 0, 0, 0, 0, -1, 0, 0],
....:         [0, 0, 0, 0, 0, 0, 0, -1, 0], [0, 0, 0, 0, 0, 0, 0, 0, -1],
....:         [-1, -1, -1, -1, -1, -1, -1, -1, -1], [3, 1, 0, 0, 0, 0, 0, 0, 0],
....:         [4, 0, 1, 0, 0, 0, 0, 0, 0], [6, 0, 0, 1, 0, 0, 0, 0, 0],
....:         [8, 0, 0, 0, 1, 0, 0, 0, 0], [6, 0, 0, 0, 0, 1, 0, 0, 0],
....:         [4, 0, 0, 0, 0, 0, 1, 0, 0], [2, 0, 0, 0, 0, 0, 0, 1, 0],
....:         [0, 0, 0, 0, 0, 0, 0, 0, 1]]
sage: P = Polyhedron(ieqs=ieqs, backend='normaliz')
sage: P.integral_points()
python2: ./libnormaliz/matrix.cpp:2901: libnormaliz::Matrix<Integer> libnormaliz::Matrix<Integer>::LLL_red(libnormaliz::Matrix<Integer>&, libnormaliz::Matrix<Integer>&) const [with Integer = double]: Assertion `(int) rank()==n' failed.
------------------------------------------------------------------------
/home/travis/sage-build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x574b)[0x7fa5afdf274b]
/home/travis/sage-build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x57b5)[0x7fa5afdf27b5]
/home/travis/sage-build/local/lib/python2.7/site-packages/cysignals/signals.so(+0x849a)[0x7fa5afdf549a]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fa5bcd37390]
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fa5bc991428]
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fa5bc99302a]
/lib/x86_64-linux-gnu/libc.so.6(+0x2dbd7)[0x7fa5bc989bd7]
/lib/x86_64-linux-gnu/libc.so.6(+0x2dc82)[0x7fa5bc989c82]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZNK11libnormaliz6MatrixIdE7LLL_redERS1_S2_+0x5b9)[0x7fa47314f4a9]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZNK11libnormaliz6MatrixIdE17LLL_red_transposeERS1_S2_+0x83)[0x7fa47314fdc3]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZN11libnormaliz15LLL_coordinatesIxdEENS_25Sublattice_RepresentationIT_EERKNS_6MatrixIT0_EE+0x56)[0x7fa4730cfde6]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZN11libnormaliz31LLL_coordinates_without_1st_colIxxEEvRNS_25Sublattice_RepresentationIT_EENS_6MatrixIT0_EES7_b+0x291)[0x7fa4730d0881]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZN11libnormaliz14ProjectAndLiftIxxE7computeEbb+0x194)[0x7fa473104e54]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZN11libnormaliz4ConeI10__gmp_exprIA1_12__mpz_structS3_EE16project_and_liftERNS_14ConePropertiesERNS_6MatrixIS4_EERKS9_SA_b+0x839)[0x7fa473179b69]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZN11libnormaliz4ConeI10__gmp_exprIA1_12__mpz_structS3_EE31try_approximation_or_projectionERNS_14ConePropertiesE+0xf00)[0x7fa4731f5370]
/home/travis/sage-build/local/lib/libnormaliz.so.3(_ZN11libnormaliz4ConeI10__gmp_exprIA1_12__mpz_structS3_EE7computeENS_14ConePropertiesE+0x1e1)[0x7fa4731e8671]
/home/travis/sage-build/local/lib/python2.7/site-packages/PyNormaliz_cpp.so(_Z14_NmzResultImplI10__gmp_exprIA1_12__mpz_structS2_EEP7_objectPN11libnormaliz4ConeIT_EES5_+0x75)[0x7fa4734f6cc5]
/home/travis/sage-build/local/lib/python2.7/site-packages/PyNormaliz_cpp.so(_Z10_NmzResultP7_objectS0_+0x150)[0x7fa4734ed890]

Originally found by

sage: RC = RiggedConfigurations(['E',8,1], [[3,2]])
sage: len(RC.module_generators)
python2: ./libnormaliz/matrix.cpp:2901: libnormaliz::Matrix<Integer> libnormaliz::Matrix<Integer>::LLL_red(libnormaliz::Matrix<Integer>&, libnormaliz::Matrix<Integer>&) const [with Integer = double]: Assertion `(int) rank()==n' failed.

(Mathematical note, through trying to find a smaller example, I am noticing that E8(1) is special in that it is the only place where I see non-full-dimensional non-empty polytopes in my computations. Well, I also see a 0-dimensional polytope for type D4(3), but that doesn't cause a crash as it is likely special-cased.)


Also can you update the comment before --enable-flint to also say you enable flint as it is part of the standard packages of Sage?

comment:13 Changed 5 years ago by git

Commit: f2137cc5bb49278657ce23f0f8c775400b494023b8f7abc04085155225e54ad168ce4a7884fdd6cf

Branch pushed to git repo; I updated commit sha1. New commits:

b8f7abcnormaliz/spkg-install: Explain --enable-flint

comment:14 Changed 5 years ago by Matthias Köppe

I can confirm this crash. Here on Mac OS X, this actually becomes an unhandled SIGABRT:

Assertion failed: ((int) rank()==n), function LLL_red, file ./libnormaliz/matrix.cpp, line 2901.
------------------------------------------------------------------------
0   signals.so                          0x000000010af572d8 print_backtrace + 40
1   ???                                 0x000000010dcac1e0 0x0 + 4526359008
------------------------------------------------------------------------
Unhandled SIGABRT: An abort() occurred.
This probably occurred because a *compiled* module has a bug
in it and is not properly wrapped with sig_on(), sig_off().
Python will now terminate.

comment:15 Changed 5 years ago by Matthias Köppe

Upstream master branch (8db5b68491a105b9185a01c29c2f126f5115e238) has different behavior but does not fix this.

comment:16 Changed 5 years ago by Thierry Monteil

Reviewers: Travis ScrimshawTravis Scrimshaw, Thierry Monteil

I confirm that the packages build and pass self-tests and doctests correctly on both 64 and 32 bit architectures on Debian stretch. I also comfirm that there is a crash on both architectures regarding the test provided by Travis (which should be added in our doctests, once the issue is fixed).

comment:17 Changed 5 years ago by Winfried Bruns

I am not sure what the matrix ieqs represents. My interpretation: inhomogeneous inequalities, and we want to compute the lattice points in the polytope they define. Under this assumption I have made a Normaliz input file rash.in

amb_space auto
inhom_inequalities
[[0, 2, 0, -1, 0, 0, 0, 0, 0],
...
[0, 0, 0, 0, 0, 0, 0, 0, 1]]
HilbertBasis

Then a run of

normaliz -c crash.in

finishes without any problems in 3.5.1 and the intended 3.5.2. It turns out that the polytope is empty:

0 module generators
1252 Hilbert basis elements of recession monoid
0 vertices of polyhedron
39 extreme rays of recession cone
11 support hyperplanes of polyhedron (homogenized)

embedding dimension = 9
affine dimension of the polyhedron = -1
rank of recession monoid = 8
...

Is my interpretation of the input correct?

comment:18 Changed 5 years ago by Winfried Bruns

Of course, if the matrix represents inhomogeneous inequalities, what is the RHS? My input above assumes it is the last coordinate. I will now try with the first.

comment:19 Changed 5 years ago by Winfried Bruns

Unfortunately Normaliz crashes when coordinate 1 is interpreted as the RHS. Will take care of it.

comment:20 Changed 5 years ago by Winfried Bruns

The problem is solved. Replace line 323 of source/libnormaliz/sublattice_representation.h

by

    if(Vertices.nr_of_rows()==0 || Vertices.rank()<EmbDim){ // use Supps for LLL coordinates

i.e., delete "-1" in this line.

comment:21 in reply to:  20 Changed 5 years ago by Travis Scrimshaw

Description: modified (diff)
Report Upstream: N/AReported upstream. Developers acknowledge bug.

Replying to Winfried:

The problem is solved. Replace line 323 of source/libnormaliz/sublattice_representation.h

by

    if(Vertices.nr_of_rows()==0 || Vertices.rank()<EmbDim){ // use Supps for LLL coordinates

i.e., delete "-1" in this line.

Great, thank you for figuring it out. I am guessing you are in the process of making a commit upstream or should I do a pull request?

Note: See TracTickets for help on using tickets.