Opened 7 months ago

Closed 6 months ago

Last modified 6 months ago

#27731 closed defect (fixed)

upgrade normaliz to 3.7.2, pynormaliz to 2.5

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-8.8
Component: packages: optional Keywords:
Cc: Winfried, jipilab, gh-sebasguts, mkoeppe, vdelecroix, tscrim Merged in:
Authors: Matthias Koeppe Reviewers: Vincent Delecroix
Report Upstream: N/A Work issues:
Branch: 0a8c6f8 (Commits) Commit:
Dependencies: #27782 Stopgaps:

Description (last modified by mkoeppe)

Another update of normaliz, pynormaliz. The upgrade of PyNormaliz from 2.1 to 2.5 fixes a critical data conversion bug (Check overflow for int conversion correctly) that was present in all previous versions.

Tarballs - see wget commands below.


Step by step instructions if you want to try it out

  1. Go to the Sage source tree and pull the branch associated to this ticket
  1. Get the tarballs and put them in the upstream repository
    $ wget -P upstream https://github.com/Normaliz/Normaliz/releases/download/v3.7.2/normaliz-3.7.2.tar.gz
    $ wget -P upstream https://files.pythonhosted.org/packages/31/8f/a1e8e489150a570c81b427d351a29b868df57015f13c12e8e6d2d2d55c45/PyNormaliz-2.5.tar.gz
    
  1. Compile and install everything
    $ sage -i pynormaliz
    
  1. Run make
    $ make build
    

Change History (36)

comment:1 Changed 7 months ago by mkoeppe

  • Dependencies set to #27682

comment:2 Changed 7 months ago by mkoeppe

  • Branch set to public/27731

comment:3 Changed 7 months ago by mkoeppe

  • Cc tscrim added
  • Commit set to 3d949a2219a3a502e3374ba68dd173bfa4acffcb
  • Status changed from new to needs_review

New commits:

eaa4584package e-antic 0.1.3b0
e5c669dupgrade normaliz
14d6c83Update normaliz to 3.7.1, pynormaliz to 2.1
64f1f19fix doctest in backend_normaliz.py
3d949a2Upgrade normaliz to 3.7.2, pynormaliz to 2.2

comment:4 Changed 7 months ago by vdelecroix

Hi,

What is the point of having several open tickets concerning Normaliz/PyNormaliz? upgrade? You want the job of the release manager to become a nightmare? If Normaliz/PyNormaliz? are not mature you should not have set anything to needs review in the first place. Also, you are shipping a beta version of e-antic which is not nice either.

comment:5 Changed 7 months ago by mkoeppe

3.7.1/2.1 are stable and an improvement over the existing versions in sage. The ticket #27682 is positively reviewed already, and per release manager, positively reviewed tickets are not to be modified further.

comment:6 Changed 7 months ago by vdelecroix

  • Status changed from needs_review to needs_work

PyNormaliz provides a test suite, there should be a spkg-check.

comment:7 Changed 7 months ago by git

  • Commit changed from 3d949a2219a3a502e3374ba68dd173bfa4acffcb to b4889111d4478b8dda2e15dac58a4d725cd2f0c0

Branch pushed to git repo; I updated commit sha1. New commits:

b488911Add build/pkgs/pynormaliz/spkg-check

comment:8 Changed 7 months ago by mkoeppe

PyNormaliz? test suite currently fails with the e-antic bug-fix release 0.1.3b0 that we have in #27682.

comment:9 Changed 7 months ago by mkoeppe

  • Description modified (diff)
  • Summary changed from upgrade normaliz to 3.7.2, pynormaliz to 2.2 to upgrade normaliz to 3.7.2, pynormaliz to 2.4

comment:10 Changed 7 months ago by git

  • Commit changed from b4889111d4478b8dda2e15dac58a4d725cd2f0c0 to 5d013883725486cee52b7b247b8d0aab6d81f7d7

Branch pushed to git repo; I updated commit sha1. New commits:

5d01388Upgrade pynormaliz to 2.4

comment:11 Changed 7 months ago by mkoeppe

  • Description modified (diff)
  • Status changed from needs_work to needs_review
  • Type changed from enhancement to defect

comment:12 Changed 7 months ago by mkoeppe

  • Description modified (diff)

comment:13 Changed 7 months ago by mkoeppe

  • Description modified (diff)
  • Summary changed from upgrade normaliz to 3.7.2, pynormaliz to 2.4 to upgrade normaliz to 3.7.2, pynormaliz to 2.5

comment:14 Changed 7 months ago by git

  • Commit changed from 5d013883725486cee52b7b247b8d0aab6d81f7d7 to a4e597762cf0ed306b81a2ab14aab576aeee85b2

Branch pushed to git repo; I updated commit sha1. New commits:

a4e5977Upgrade pynormaliz to 2.5

comment:15 Changed 7 months ago by vdelecroix

  • Reviewers set to Vincent Delecroix
  • Status changed from needs_review to needs_work

With Sage built with Python3

sage -t --long polyhedron/backend_normaliz.py
**********************************************************************
File "polyhedron/backend_normaliz.py", line 497, in sage.geometry.polyhedron.backend_normaliz.Polyhedron_normaliz._normaliz_format
Failed example:
    P = Polyhedron(vertices=[[0, 0], [0, 1], [1, 0]], # indirect doctest; optional - pynormaliz
                   backend='normaliz', verbose=True)
Exception raised:
    Traceback (most recent call last):
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 671, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1095, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.geometry.polyhedron.backend_normaliz.Polyhedron_normaliz._normaliz_format[0]>", line 2, in <module>
        backend='normaliz', verbose=True)
      File "sage/misc/lazy_import.pyx", line 354, in sage.misc.lazy_import.LazyImport.__call__ (build/cythonized/sage/misc/lazy_import.c:3690)
        return self.get_object()(*args, **kwds)
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/geometry/polyhedron/constructor.py", line 604, in Polyhedron
        return parent(Vrep, Hrep, convert=convert, verbose=verbose)
      File "sage/structure/parent.pyx", line 902, in sage.structure.parent.Parent.__call__ (build/cythonized/sage/structure/parent.c:9224)
        return mor._call_with_args(x, args, kwds)
      File "sage/structure/coerce_maps.pyx", line 181, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_with_args (build/cythonized/sage/structure/coerce_maps.c:5074)
        raise
      File "sage/structure/coerce_maps.pyx", line 176, in sage.structure.coerce_maps.DefaultConvertMap_unique._call_with_args (build/cythonized/sage/structure/coerce_maps.c:4962)
        return C._element_constructor(x, *args, **kwds)
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/geometry/polyhedron/parent.py", line 517, in _element_constructor_
        return self.element_class(self, Vrep, Hrep, **kwds)
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_normaliz.py", line 156, in __init__
        Polyhedron_base.__init__(self, parent, Vrep, Hrep, **kwds)
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/geometry/polyhedron/base.py", line 124, in __init__
        self._init_from_Vrepresentation(vertices, rays, lines, **kwds)
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_normaliz.py", line 263, in _init_from_Vrepresentation
        self._init_from_normaliz_data(data, verbose=verbose)
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_normaliz.py", line 201, in _init_from_normaliz_data
        print(self._normaliz_format(data), end='')
      File "/opt/sage/sage-py3-gcc/local/lib/python3.7/site-packages/sage/geometry/polyhedron/backend_normaliz.py", line 511, in _normaliz_format
        for key, value in data.iteritems():
    AttributeError: 'dict' object has no attribute 'iteritems'
**********************************************************************

comment:16 Changed 7 months ago by git

  • Commit changed from a4e597762cf0ed306b81a2ab14aab576aeee85b2 to d44a00d9b3b8b7ee6e4b1d89bf8adbcedebdedf3

Branch pushed to git repo; I updated commit sha1. New commits:

292e01fbackend_normaliz: py3 fix
d44a00dbackend_normaliz: Canonicalize sort order of fields in normaliz files

comment:17 Changed 7 months ago by mkoeppe

  • Status changed from needs_work to needs_review

comment:18 Changed 7 months ago by mkoeppe

(The canonicalization of the output files is a good idea anyway and takes care of differences in dictionary orders between py2 and py3.)

comment:19 Changed 7 months ago by jipilab

Compiles cleanly on debian.

Running tests using pynormaliz in the folder src/sage/geometry/polyhedron:

$ sage -t *.py
Using --optional=bliss,dochtml,e_antic,gfortran,lrslib,memlimit,mpir,ninja_build,normaliz,pynormaliz,python2,sage,topcom
Doctesting 25 files.
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------

and without:

$ sage -t *.py --optional=bliss,dochtml,gfortran,lrslib,memlimit,mpir,ninja_build,python2,sage,topcom
Using --optional=bliss,dochtml,gfortran,lrslib,memlimit,mpir,ninja_build,python2,sage,topcom
Doctesting 25 files.
----------------------------------------------------------------------
All tests passed!
----------------------------------------------------------------------

comment:20 Changed 7 months ago by jipilab

Merge conflict with the new beta.

comment:21 Changed 7 months ago by git

  • Commit changed from d44a00d9b3b8b7ee6e4b1d89bf8adbcedebdedf3 to dd9a81bfe6660f52778b4becbc8f848e04f3b07e

Branch pushed to git repo; I updated commit sha1. New commits:

dd9a81bMerge tag '8.8.beta4' into t/27731/public/27731

comment:22 Changed 7 months ago by git

  • Commit changed from dd9a81bfe6660f52778b4becbc8f848e04f3b07e to 0a8c6f867f6779020df69e8687f32edf70a778f7

Branch pushed to git repo; I updated commit sha1. New commits:

9b3a5baFix error message in doctest for new pynormaliz version
0a8c6f8Merge branch 't/27782/1_pynormaliz_doctest_failing_in_src_sage_geometry_polyhedron_backend_normaliz_py' into t/27731/public/27731

comment:23 Changed 7 months ago by mkoeppe

  • Dependencies changed from #27682 to #27782

comment:24 Changed 7 months ago by mkoeppe

Needs review again!

comment:25 Changed 7 months ago by vdelecroix

  • Status changed from needs_review to positive_review

comment:26 Changed 6 months ago by vbraun

  • Branch changed from public/27731 to 0a8c6f867f6779020df69e8687f32edf70a778f7
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:27 follow-up: Changed 6 months ago by slabbe

  • Commit 0a8c6f867f6779020df69e8687f32edf70a778f7 deleted

As I reported on sage-release for 8.8.beta6, I get troubles installing pynormaliz.

Is this normal doctor?

comment:28 in reply to: ↑ 27 Changed 6 months ago by vdelecroix

Replying to slabbe:

As I reported on sage-release for 8.8.beta6, I get troubles installing pynormaliz.

Is this normal doctor?

Could you check whether there is a config.log in

/home/slabbe/GitBox/sage/local/var/tmp/sage/build/pynormaliz-2.5

and if so attach it.

comment:29 Changed 6 months ago by slabbe

No config.log:

$ ls /home/slabbe/GitBox/sage/local/var/tmp/sage/build/pynormaliz-2.5
checksums.ini  package-version.txt  spkg-install  src
dependencies   spkg-check           SPKG.txt      type

comment:30 follow-up: Changed 6 months ago by mkoeppe

It would of course be in the src subdirectory.

comment:31 follow-up: Changed 6 months ago by slabbe

The complete pynormaliz-2.5.log file is pasted on framabin here

In particular, there is this line:

    configure: error: source directory already configured; run "make distclean" there first

Maybe this has to do with the fact that my brother asked me to try to test this ticket one month ago where I installed a previous version of those packages?

Last edited 6 months ago by slabbe (previous) (diff)

comment:32 in reply to: ↑ 30 ; follow-up: Changed 6 months ago by slabbe

Replying to mkoeppe:

It would of course be in the src subdirectory.

No config.log file in the src directory, but there is a config.py.in:

$ cd /home/slabbe/GitBox/sage/local/var/tmp/sage/build/pynormaliz-2.5/src
$ ls
config.py.in  doc       m4                  PyNormaliz.py  setup.py
configure     examples  NormalizModule.cpp  Readme.md      tests
COPYING       GPLv2     PKG-INFO            setup.cfg
$ cat config.py.in 
# Configuration variables
ENFNORMALIZ=@ENFNORMALIZ@
Last edited 6 months ago by slabbe (previous) (diff)

comment:33 in reply to: ↑ 31 Changed 6 months ago by vdelecroix

Replying to slabbe:

The complete pynormaliz-2.5.log file is pasted on framabin here

I noticed. And this is not what I asked for.

comment:34 in reply to: ↑ 32 ; follow-up: Changed 6 months ago by vdelecroix

Replying to slabbe:

Replying to mkoeppe:

It would of course be in the src subdirectory.

No config.log file in the src directory, but there is a config.py.in:

$ cd /home/slabbe/GitBox/sage/local/var/tmp/sage/build/pynormaliz-2.5/src
$ ls
config.py.in  doc       m4                  PyNormaliz.py  setup.py
configure     examples  NormalizModule.cpp  Readme.md      tests
COPYING       GPLv2     PKG-INFO            setup.cfg
$ cat config.py.in 
# Configuration variables
ENFNORMALIZ=@ENFNORMALIZ@

That is to be expected. The setup.py script calls $ sh configure that creates the config.py from the template file config.py.in. In your situation, the process get stuck at the $ sh configure call (as you pynormaliz-2.5.log already mentioned).

Do you have sh installed?

BTW: we should not discuss this on this close ticket... could you open a thread on sage-devel or so.

comment:35 in reply to: ↑ 34 Changed 6 months ago by slabbe

BTW: we should not discuss this on this close ticket... could you open a thread on sage-devel or so.

OK, let's continue here: https://groups.google.com/d/msg/sage-devel/2virhaHdt1k/nK-AtlByBAAJ

comment:36 Changed 6 months ago by mkoeppe

Follow-up in #27920.

Note: See TracTickets for help on using tickets.