Opened 9 years ago

Closed 9 years ago

Last modified 9 years ago

#10039 closed enhancement (fixed)

Make Parma Polyhedra Library a standard library

Reported by: vbraun Owned by: mhampton
Priority: major Milestone: sage-4.7
Component: geometry Keywords: ppl spkg
Cc: novoselt, jdemeyer, fbissey Merged in: sage-4.7.alpha4
Authors: Volker Braun Reviewers: Marshall Hampton, Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: Commit:
Dependencies: Stopgaps:

Description (last modified by jdemeyer)

The Parma Polyhedra Library (ppl) is for many workloads the fastest library for polyhedral computations. It is also high-quality code, for example GCC uses it (optionally) to optimize loops.

  • ppl has already been widely tested on a multitude of platforms.
  • Native C++ with a pure C interface.
  • Contains a huge testsuite _and_ passes its own testsuite (in contrast to some other polydedral library that shall remain unnamed)

Official webpage: http://www.cs.unipr.it/ppl/

My plan is to

  1. Create a PPL spkg.
  2. Write a Cython interface.
  3. Base sage.geometry.cone.Cone on PPL instead of Polyhedron/cddlib
  4. Split sage.geometry.polyhedra.Polyhedron into an abstract base class and derived classes that use different polyhedral computation libraries.

Current status:

  1. My cython wrapper for PPL is attached. It has full doctest coverage and any invalid input is caught and raises ValueError.
  2. Is split off into trac #10140.

For convenience I mirrored the reference manual page here: http://www.stp.dias.ie/~vbraun/Sage/html/en/reference/sage/libs/ppl.html

To apply this ticket:

Attachments (12)

ppl-0.11.p0.log (437.3 KB) - added by drkirkby 9 years ago.
Log showing how some of the test code failed to build when SPKG_CHECK=yes. Without that, it builds OK in < 82 seconds. This is on a Sun Ultra 27 running OpenSolaris? 06/2009
spkg-install.patch (257 bytes) - added by vbraun 9 years ago.
diff for spkg/install
install (8.9 KB) - added by vbraun 9 years ago.
Updated spkg-install for sage-4.6.alpha1
deps (20.5 KB) - added by vbraun 9 years ago.
Updated spkg/standard/deps file with PPL added.
spkg-standard-deps.patch (764 bytes) - added by vbraun 9 years ago.
diff for spkg/standard/deps
trac_10039_scripts_hgignore.patch (526 bytes) - added by vbraun 9 years ago.
Add two ppl binaries to $SAGE_LOCAL/bin/.hgignore
trac_10039_parma_polyhedra_library.patch (186.4 KB) - added by vbraun 9 years ago.
Updated patch
trac_10039_root_repo.patch (1.6 KB) - added by vbraun 9 years ago.
Patch to SAGE_ROOT repo
trac_10039_SAGE_LOCAL_bin_repo.patch (508 bytes) - added by vbraun 9 years ago.
patch to SAGE_LOCAL/bin repo
10039_manifest.patch (671 bytes) - added by jdemeyer 9 years ago.
Patch to MANIFEST.in
trac_10039_ppl_fix_extremize.2.patch (5.1 KB) - added by vbraun 9 years ago.
Updated patch
trac_10039_ppl_fix_extremize.patch (5.1 KB) - added by vbraun 9 years ago.
Updated patch

Download all attachments as: .zip

Change History (99)

comment:1 Changed 9 years ago by mhampton

Rather than retire cddlib, it would be better I think to have cddlib as an optional backend, along with lrs. If ppl is consistently better than those then it could be the default.

comment:2 Changed 9 years ago by novoselt

If there will be a Cython interface to PPL, it definitely should become the default backend as it must be drastically faster on simple cases then the current situation. And for "complicated" cases users should be willing to spend some time to learn about advantages of different systems and pick the right one, if speed becomes an issue.

comment:3 Changed 9 years ago by mhampton

This package installed fine on Kubuntu 10.04 and OS X 10.6. It does take a while to compile - 10 minutes on the first machine and 15 on the other.

comment:4 Changed 9 years ago by vbraun

Yes, templated C++ code is really giving the compiler something to do.

But if you think thats slow then don't run the testsuite (SAGE_CHECK=yes)...

comment:5 Changed 9 years ago by mhampton

One reason I gave up any hope of making polymake standard was the compilation time; it was one reason I began writing polyhedra.py instead of improving the polymake interface.

Are there many things in PPL that go beyond cddlib? I am having trouble wading through their documentation so its unclear to me.

comment:6 Changed 9 years ago by vbraun

  • Cc novoselt added
  • Description modified (diff)
  • Summary changed from Make Parma Polyhedra Library a standard package to Make Parma Polyhedra Library a standard library

comment:7 Changed 9 years ago by novoselt

I read the module documentation and it looks quite nice, although the PPL interface and their documentation do not seem to be very clear to an unfamiliar user (as Marshall has already noted above). So if eventually Sage front-end is structured more like current geometric classes, it would be better - I guess that's exactly your plan in 4.

Can PPL compute integral points inside a bounded polytope?

comment:8 Changed 9 years ago by vbraun

Just to be clear, the PPL wrapper is not going to be exposed to the end user by default. She/he is supposed to use the higher-level Polyhedron and Cone classes.

The PPL sports grids (lattices defined through generators or congruences) and a MIP solver but, as far as I know, no function that directly enumerates grid points (i.e. the intersection of a grid with a polyhedron). Though it probably would not be too difficult to write one with these building blocks. In any case, I haven't cython-wrapped grids or MIPs so far.

comment:9 Changed 9 years ago by vbraun

  • Description modified (diff)

comment:10 Changed 9 years ago by vbraun

  • Description modified (diff)

comment:11 follow-up: Changed 9 years ago by mhansen

This does build successfully on Cygwin, but it does take _quite_ awhile:

real    346m56.531s
user    22m33.908s
sys     26m33.122s
Successfully installed ppl-0.11.p0

comment:12 Changed 9 years ago by vbraun

I take it you are not running the test suite? That one is quite extensive...

The 22 minutes in user space could be realistic for a slow machine. The 26min is sys is a bit suspicious, are you running on swap? But the real question is, why did your computer spend 6h walltime for 40 minutes of work?

I just tried the ppl spkg in an opensolaris virtual machine (single core) and it takes about 7 minutes walltime, 5m user, 1m sys.

comment:13 Changed 9 years ago by vbraun

I've removed all language interfaces (except the native C++), this shaves off a few minutes. Now on my Fedora 13 box normal compilation takes 1 minute and the testsuite 20 minutes:

[vbraun@volker-desktop spkg]$ /usr/bin/time sage -f ppl-0.11.p0.spkg 
...
35.84user 17.68system 1:02.23elapsed 86%CPU (0avgtext+0avgdata 229008maxresident)k
4240inputs+1152096outputs (37major+5069087minor)pagefaults 0swaps

Testing:

[vbraun@volker-desktop ~]$ SAGE_CHECK=yes time sage -f ppl-0.11.p0.spkg
...
771.04user 281.70system 20:37.31elapsed 85%CPU (0avgtext+0avgdata 453712maxresident)k
13710920inputs+46999048outputs (91major+76885818minor)pagefaults 0swaps

comment:14 in reply to: ↑ 11 Changed 9 years ago by drkirkby

Replying to mhansen:

This does build successfully on Cygwin, but it does take _quite_ awhile:

real    346m56.531s
user    22m33.908s
sys     26m33.122s
Successfully installed ppl-0.11.p0

That sure is a long time. I think it's because autoconf scripts takes ages on Cygwin, because Windows has no fork.

On my Sun Ultra 27, it takes less than 82 seconds to install.

real	1m21.460s
user	4m6.543s
sys	0m27.474s
Successfully installed ppl-0.11.p0
Now cleaning up tmp files.
Making Sage/Python scripts relocatable...
Making script relocatable
Finished installing ppl-0.11.p0.spkg

However, it fails the test suite. To be more precise, the tests it runs pass, but it fails to build some of the code needed for the tests.

I'm not sure what the gcc option -W is supposed to do. According to the gcc manual -w (lower case) is to suppress all warnings. If -W does the same as -w, then it seems the code tries to supporess the warnings, then enable them, which seems a bit illogical. But I can't find out exactly what -W is supposed to do.

BTW, this should be called ppl-0.11 and not ppl-0.11.p0. Only once a version has appeared in Sage, and patches have been applied, should it be called .p0.

A couple more points.

  • There is no code to handle 64-bit builds on platforms where SAGE64=yes. It needs something like this:
        if [ "x$CFLAG64" = x ]; then 
            CFLAG64=-m64
        fi
    
        # If the environment variable SAGE64=yes, force building a 64-bit version:
        if [ "x$SAGE64" = xyes ]; then
            echo "Building a 64-bit version of ppl"
            CC="$CC $CFLAG64"
            export CC
        fi
    

That needs to be in both spkg-install and spkg-check.

  • I don't think Unpack distribution into src/ directory needs to be in the Special Update/Build Instructions section, as that is the same with every single package.

  • I would start spkg-check with #!/usr/bin/env bash to be consistent. That said, there's nothing there which should not work with any half-reasonable shell.
  • It would be better to use $MAKE in spkg-check so the tests can be run in parallel too, which should speed things up a lot.

Changed 9 years ago by drkirkby

Log showing how some of the test code failed to build when SPKG_CHECK=yes. Without that, it builds OK in < 82 seconds. This is on a Sun Ultra 27 running OpenSolaris? 06/2009

comment:15 Changed 9 years ago by vbraun

  • Description modified (diff)

The testsuite fails on Solaris gcc 4.5.0, most likely due to a gcc bug. For reference, the real error is

interval1.cc:76:3: error: no matching function for call to 'Parma_Polyhedra_Library::Interval<float, Parma_Polyhedra_Library::Interval_Restriction_None<Parma_Polyhedra_Library::Interval_Info_Bitset<unsigned int, <unnamed>::My_Interval<float>::Floating_Point_Real_Interval_Info_Policy> > >::join_assign(double)'
interval1.cc:194:3:   instantiated from here

Everything should work fine with gcc 4.5.1. In particular, I could compile and run the testsuite on OpenSolaris?/i386 with gcc 4.5.1. Also note that the offending floating point code is currently not used in the Cython wrapper. A similar bug report is here: https://www.cs.unipr.it/mantis/view.php?id=110

About the -W, the gcc manual says:

-Wextra: This enables some extra warning flags that are not enabled by -Wall. (This option used to be called -W. The older name is still supported, but the newer name is more descriptive.)

I have addressed the other issues that you raised. The updated spgk is here:

http://www.stp.dias.ie/~vbraun/Sage/spkg/ppl-0.11.spkg

comment:16 Changed 9 years ago by vbraun

Sorry, I was too quick. The testsuite does not like "MAKE=make -j8", there is a missing shell escape. So I'll leave it at "make check" in the spkg-check for now. I've reported this upstream at https://www.cs.unipr.it/mantis/view.php?id=118

I'm also getting the testsuite "no matching function call" from Opensolaris/gcc 4.5.1.

comment:17 Changed 9 years ago by vbraun

I have patched out the PPL testsuite for Box<>, BD_Shape<>, and Octagonal_Shape<>. I think these templated classes are newer and not as well-tested. Note that the polyhedron code does not use templates. In any case, they are not currently exposed by the Cython wrapper and they don't provide any functionality that we need in the near future.

The updated spgk now compiles a) in parallel and b) under Solaris/gcc 4.5.1 without errors, including the testsuite.

As usual, get it here: http://www.stp.dias.ie/~vbraun/Sage/spkg/ppl-0.11.spkg

comment:18 follow-up: Changed 9 years ago by novoselt

Some notes based on the posted documentation:

  1. "Note that, by convention, every polyhedron must contain a point" can be clarified a little that only generator systems containing a point may define a polyhedron, since there is no point issue in defining a polyhedron by a constraint system. (I would also add that it is natural to think about "just rays" as starting at the origin, but since your goal is to wrap PPL as closely as possible, it is not your design decision.)
  2. In the documentation of Generator I see "frac"s instead of fractions.
  3. add_space_dimensions_and_embed and add_space_dimensions_and_project - I think you want (x,y)\in P, not R^2. Also the top of the documentation for "project" still says "embed". (I also find names confusing, but I guess that's how PPL calls them.)
  4. Why affine_dimension of the empty polyhedron is 0? Shouldn't it be -1 to distinguish it from points?
  5. Am I right that "concatenating" polyhedra means taking their Cartesian product? Perhaps, it is a simpler explanation than in concatenate_assign?
  6. contains_integer_point has wrong documentation description about being bounded.
  7. I don't understand what exactly is done by drop_some_non_integer_points.
  8. frequency does not have documentation.
  9. Is it necessary to have hash_code, especially if it only returns the dimension? My concern is that it can be confusing how it is related to standard hash in Python, also it seems strange to use it for mutable objects (again, because that's not how things work in Python). If this function is not used anywhere, I would suggest removing it.
  10. I got upset when I read in the documentation of max_space_dimension that it is bounded, but the doctest immediately made me happy ;-)
  11. clear_mpz_globals, gmp_randrange, init_mpz_globals have no documentation.

comment:19 in reply to: ↑ 18 Changed 9 years ago by vbraun

  • Status changed from new to needs_review

I fixed the documentation according to your suggestions.

  1. Why affine_dimension of the empty polyhedron is 0? Shouldn't it be -1 to distinguish it from points?

The internal representation of dimensions is an unsigned int. So PPL returns 0 instead of something negative. Using signed integer would half the max_space_dimension() ;-)

  1. Am I right that "concatenating" polyhedra means taking their Cartesian product? Perhaps, it is a simpler explanation than in concatenate_assign?

Yes. I just copy/pasted the C++ documentation. I've added a line that this is the Cartesian product.

  1. I don't understand what exactly is done by drop_some_non_integer_points.

All that is guaranteed is that the result is non-strictly contained in the original polyhedron and that integral vertices survive. Its not the most useful function, but it is there and was easy to wrap ;-)

  1. frequency does not have documentation.

I removed this one, it is only useful for grids (spanned lattices / congruences).

  1. Is it necessary to have hash_code, especially if it only returns the dimension? My concern is that it can be confusing how it is related to standard hash in Python, also it seems strange to use it for mutable objects (again, because that's not how things work in Python). If this function is not used anywhere, I would suggest removing it.

Removed.

  1. I got upset when I read in the documentation of max_space_dimension that it is bounded, but the doctest immediately made me happy ;-)

Yes, should be enough for most applications ;)

  1. clear_mpz_globals, gmp_randrange, init_mpz_globals have no documentation.

I didn't define these, they were imported from gmp.pxi and sphinx adds them to the documentation for some reason. Anyways, importing gmp.pxi is not necessary so I removed it.

comment:20 Changed 9 years ago by jdemeyer

  • Status changed from needs_review to needs_work

I have tried building with the new spkg and sagelib patch, but I get the following error:

Building modified file sage/libs/pari/gen.pyx.
Traceback (most recent call last):
  File "setup.py", line 855, in <module>
    queue = compile_command_list(ext_modules, deps)
  File "setup.py", line 815, in compile_command_list
    dep_file, dep_time = deps.newest_dep(f)
  File "setup.py", line 722, in newest_dep
    for f in self.all_deps(filename):
  File "setup.py", line 703, in all_deps
    for f in self.immediate_deps(filename):
  File "setup.py", line 685, in immediate_deps
    self._deps[filename] = self.parse_deps(filename)
  File "setup.py", line 675, in parse_deps
    raise IOError, "could not find dependency %s included in %s."%(path, filename)
IOError: could not find dependency ../../local/include/ppl.hh included in sage/libs/ppl.pyx.
sage: There was an error installing modified sage library code.

ERROR installing SAGE

comment:21 Changed 9 years ago by vbraun

  • Cc jdemeyer added
  • Description modified (diff)
  • Status changed from needs_work to needs_review

Is SAGE_LOCAL!=SAGE_ROOT/local in your install? It should work in either case, of course. I looked into how cython discovers the dependencies and the problem is that the setup.py expects the extension ".h" for C/C++ headers. But PPL uses ppl.hh. I fixed the dependency discovery for cython in #10233, which is a new prerequisite for this patch.

I also changed the ppl wrapper (this ticket) to rely on the search path instead of cdef extern from "../../local/include/ppl.hh", which I probably should have done from the beginning.

comment:22 follow-up: Changed 9 years ago by jdemeyer

If you want to make your package a standard package, you also need to patch spkg/install and spkg/standard/deps (most likely, that explains my error, your package simply wasn't installed).

comment:23 Changed 9 years ago by jdemeyer

  • Status changed from needs_review to needs_work

comment:24 in reply to: ↑ 22 Changed 9 years ago by drkirkby

Replying to jdemeyer:

If you want to make your package a standard package, you also need to patch spkg/install and spkg/standard/deps (most likely, that explains my error, your package simply wasn't installed).

Any attempt to make a package standard would need discussion on sage-devel - which basically means William would have to agree to it. And I know William is not keen on adding too many standard packages. That's not to say he would not make it standard, but whereas a year or two ago virtually anything could become standard, that is no longer so.

Dave

comment:25 follow-up: Changed 9 years ago by vbraun

  • Status changed from needs_work to needs_review

I'll attach updated spkg/install and spkg/standard/deps files. The only dependency is mpir, so its pretty simple.

I did start a discussion thread on sage-devel about making it a standard spkg here: http://groups.google.com/group/sage-devel/browse_frm/thread/46e4363245607218 Nobody objected and Andrey, Dima, Marshall, and I were in favor.

Changed 9 years ago by vbraun

diff for spkg/install

Changed 9 years ago by vbraun

Updated spkg-install for sage-4.6.alpha1

comment:26 Changed 9 years ago by fbissey

I think there are a few more things to do for it to be standard sage material. In module_list.py you may want to do a bit more:

Extension('sage.libs.ppl', 
 	  sources = ['sage/libs/ppl.pyx', 'sage/libs/ppl_shim.cc'],
 	  libraries = ['ppl', 'gmpxx', 'gmp', 'm'], 
 	  language="c++",
          depends = [SAGE_LOCAL + "/include/ppl.hh"]),

In your wrapper sage/libs/ppl.pyx, ../../local/include/ppl.hh should be simplified to ppl.hh, same for everything else that has ../../local/include, setup.py knows to include this folder. You only needs to add things if it is in a sub-folder and then you do that by adding an include line in module_list.py.

comment:27 in reply to: ↑ 25 Changed 9 years ago by drkirkby

Replying to vbraun:

I did start a discussion thread on sage-devel about making it a standard spkg here: http://groups.google.com/group/sage-devel/browse_frm/thread/46e4363245607218 Nobody objected and Andrey, Dima, Marshall, and I were in favor.

I think you should get William's approval too. I very much doubt it should be standard without being optional first. There's a ticket open to clarify that, but I can't be bothered to look for it now.

Basically, getting a few people to agree to make it standard is not enough.

Dave

comment:28 Changed 9 years ago by mhampton

The bar should be high for standard packages, but not unreasonably so. Don't make it difficult just so its difficult.

Most packages should be optional before being standard. But if a lot of functionality is tied to them being present, as is the case here, then that doesn't always make sense.

Cddlib would never make it as a standard package now. This is a big step towards being able to remove cddlib.

While William certainly has veto power, he did read that thread and did not object. At this point I think that after someone gives this a positive review, its up to the release manager at the time to make the call.

comment:29 Changed 9 years ago by vbraun

I renamed the include ../../local/include/ppl.hh to ppl.hh. Note that the ppl.pyx now really depends on #10233, because the current setup.py does not know how to include *.hh headers.

I also added the explicit dependency on SAGE_LOCAL+/include/ppl.hh.

Last, I just want to point out that now is the time to add PPL. I've fixed all outstanding bugs in Polyhedron that can be fixed within the current framework, and to continue improving it we need PPL.

comment:30 Changed 9 years ago by vbraun

  • Cc fbissey added

comment:31 Changed 9 years ago by fbissey

There was no need to cc me, I have been added automatically because I commented. I didn't react immediately because I was sleeping (time zone GMT+12). Anyway the patch looks good to me now, the dep file looks right as well. I will probably include this in sage-on-gentoo in 4.6.1.alpha1 forward to give it some coverage. I may have to request a revision bump of the version of ppl shipped in gentoo first (currently shipping 0.10.2).

comment:32 Changed 9 years ago by fbissey

It has been noted in Gentoo (see http://bugs.gentoo.org/show_bug.cgi?id=346173) that ppl has an automagic dependency on glpk. that is ppl_lpsol will be built if glpk is found. It also looks like from the report that the test for ppl_lpsol succeeds or fails depending on the version of glpk used.

glpk is not a dependency you gave to ppl so some sage install may end up with ppl_lpsol while others won't, depending on build order. I suggest that either glpk is added to the dependencies of ppl or that --disable-ppl_lpsol is passed to configure.

Changed 9 years ago by vbraun

Updated spkg/standard/deps file with PPL added.

Changed 9 years ago by vbraun

diff for spkg/standard/deps

comment:33 Changed 9 years ago by vbraun

Thanks for pointing out the soft glpk dependency! I checked and the Fedora rpm requires glpk-devel, too. I've updated the deps file to include this dependency.

comment:34 Changed 9 years ago by vbraun

For the patch bot:

Apply trac_10039_parma_polyhedra_library.patch

comment:35 Changed 9 years ago by vbraun

I've now added sig_on/sig_off blocks around expensive library calls. Interrupting works, too:

sage: Cone(random_matrix(QQ,100,10).rows())
^CERROR: An unexpected error occurred while tokenizing input
The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (2152, 0))

---------------------------------------------------------------------------
KeyboardInterrupt                         Traceback (most recent call last)

/home/vbraun/opt/sage-4.6.1.rc1/devel/sage-main/<ipython console> in <module>()

/home/vbraun/Sage/sage/local/lib/python2.6/site-packages/sage/geometry/cone.pyc in Cone(rays, lattice, check, normalize)
    390             pass
    391     cone = C_Polyhedron(gs)
--> 392     return _Cone_from_PPL(cone, lattice)
    393 
    394 def _Cone_from_PPL(cone, lattice):

/home/vbraun/Sage/sage/local/lib/python2.6/site-packages/sage/geometry/cone.pyc in _Cone_from_PPL(cone, lattice)
    420     rays = []
    421     lines = []
--> 422     for g in cone.minimized_generators():
    423         ray = lattice(g.coefficients())
    424         ray.set_immutable()

KeyboardInterrupt: 

Note: in this example I have #10140: "Base sage.geometry.cone on the Parma Polyhedra Library (PPL)" applied.

comment:36 Changed 9 years ago by mhampton

This looks very good.

One doctest problem is that on 32-bit machines we get:

File "/Users/mh/sagestuff/sage-4.6.1.rc1/devel/sage-t1/sage/libs/ppl.pyx", line 2299:
    sage: C_Polyhedron(1, 'empty').max_space_dimension()
Expected:
    1152921504606846973
Got:
    357913939

so maybe that should be something like

sage: max_dim = C_Polyhedron(1, 'empty').max_space_dimension()
sage: max_dim in [357913939,1152921504606846973]
sage: True

or just

sage: max_dim = C_Polyhedron(1, 'empty').max_space_dimension()
sage: max_dim > 1
sage: True

comment:37 Changed 9 years ago by mhampton

  • Status changed from needs_review to needs_work

comment:38 Changed 9 years ago by vbraun

  • Reviewers set to Marshall Hampton
  • Status changed from needs_work to needs_review

I fixed the max_space_dimension along the lines of Marshall's suggestion. I also improved the doctesting of the ascii_dump() methods using Jeroen's test_executable() function.

comment:39 Changed 9 years ago by mhampton

OK, looks good, all doctests pass now. I will try to test on t2, then give a positive review.

comment:40 Changed 9 years ago by mhampton

I can't build this on t2, but I don't really know what I'm doing on Solaris. I do have this in my .profile:

if [ `uname -n` = t2 ] ; then
   . /usr/local/bin/t2-setup
fi 

as suggested by the login message.

I get this error:

CC Version
gcc -v
/home/mhampton/solaris/sage-4.6.1.rc1/local/bin/sage-spkg: line 319: gcc: command not found
Unable to determine C compiler version.
****************************************************
Failed to find mpir.  Please install the mpir spkg

comment:41 Changed 9 years ago by vbraun

On t2 (only), the newest_version script returns nothing unless it is run from $SAGE_ROOT/spkg, which is why you got the "Failed to find mpir" message. I updated the ppl spkg (same location), and it works now on t2 as well.

I followed the following steps to build on t2:

  • . /usr/local/bin/t2-setup
  • tar xzf sage-4.6.1-sunos-32bit-5.10-sun4v-SunOS.tar.gz
  • sage -f ppl-0.11.spkg

Changed 9 years ago by vbraun

Add two ppl binaries to $SAGE_LOCAL/bin/.hgignore

comment:42 Changed 9 years ago by fbissey

I got a weird test failure if you have an idea:

sage -t -force_lib "devel/sage/sage/libs/ppl.pyx"           
size 3 1 3 2 f -RPI_V -RPI  -NNC_V -NNC
topology NECESSARILY_CLOSED
1 x 3 (sorted)
index_first_pending 1
1 3 2 P
size 4 1 3 2 -1 f -RPI_V +RPI  -NNC_V +NNC
topology NOT_NECESSARILY_CLOSED
1 x 4 (sorted)
index_first_pending 1
-1 3 -2 -1 >
NOTHINGNOTHING**********************************************************************
File "/usr/share/sage/devel/sage/sage/libs/ppl.pyx", line 1271:
    sage: p.maximize( +x )
Expected:
    {'sup_d': 0, 'sup_n': 0, 'bounded': False, 'maximum': False, 'generator': None}
Got:
    {'sup_d': 0, 'sup_n': 0, 'bounded': False, 'maximum': True, 'generator': None}
**********************************************************************
File "/usr/share/sage/devel/sage/sage/libs/ppl.pyx", line 1340:
    sage: p.minimize( -x )
Expected:
    {'minimum': False, 'bounded': False, 'inf_d': 0, 'generator': None, 'inf_n': 0}
Got:
    {'minimum': True, 'bounded': False, 'inf_d': 0, 'generator': None, 'inf_n': 0}
space_dim 2
-ZE -EM  +CM +GM  +CS +GS  -CP -GP  -SC +SG 
con_sys (up-to-date)
topology NECESSARILY_CLOSED
2 x 3 (sorted)
index_first_pending 2
-1 3 2 =
1 0 0 >=

gen_sys (up-to-date)
topology NECESSARILY_CLOSED
2 x 3 (not_sorted)
index_first_pending 2
0 2 -3 L
2 0 1 P

sat_c
0 x 0

sat_g
2 x 2
0 0 
0 1 

**********************************************************************
File "/usr/share/sage/devel/sage/sage/libs/ppl.pyx", line 2204:
    sage: C_Polyhedron(1, 'empty').max_space_dimension()
Expected:
    1152921504606846973
Got:
    357913939
size 3 1 3 2 f -RPI_V -RPI  -NNC_V -NNC
**********************************************************************
3 items had failures:
   1 of  15 in __main__.example_29
   1 of  15 in __main__.example_30
   1 of   4 in __main__.example_48
***Test Failed*** 3 failures.
For whitespace errors, see the file /home/francois/.sage/tmp/.doctest_ppl.py
         [15.8 s]
 
----------------------------------------------------------------------
The following tests failed:


        sage -t -force_lib "devel/sage/sage/libs/ppl.pyx"
Total time for all tests: 15.8 seconds

comment:43 Changed 9 years ago by vbraun

Francois, I fixed the 32 bit issue in max_space_dimension() a while ago, so you have an old version of the patch.

About the other errors, is this on Gentoo with your own PPL library? I suggest you use the Sage spkg since there are certain configuration options that I rely on.

comment:44 Changed 9 years ago by fbissey

OK I will check the patch, it is possible that I have add it sitting there for a while before using it.

I'll have to check your configuration options but from my porting point of view it has to be conservative with what we have in Gentoo since ppl is part of the toolchain (you need it to build gcc with graphite support). I can work out something if it isn't but that will be a downright pain. I'll keep you posted on that.

comment:45 Changed 9 years ago by vbraun

Here is the PPL configuration:

./configure --prefix="$SAGE_LOCAL" --with-gmp-prefix="$SAGE_LOCAL" --enable-fpmath=no --enable-coefficients=mpz --enable-interfaces=c++ 

I need the GMP/MPIR based coefficients and C++ interface. PPL supports some tricks to work with floating point numbers as coefficients, but I'm not using that as it obviously comes with restrictions on the allowed coefficients. Also, the PPL Cython wrapper turns off the FPU rounding mode that PPL's floating point arithmetic relies on.

comment:46 Changed 9 years ago by fbissey

I still get

**********************************************************************
File "/usr/share/sage/devel/sage/sage/libs/ppl.pyx", line 1271:
    sage: p.maximize( +x )
Expected:
    {'sup_d': 0, 'sup_n': 0, 'bounded': False, 'maximum': False, 'generator': None}
Got:
    {'sup_d': 0, 'sup_n': 0, 'bounded': False, 'maximum': True, 'generator': None}
**********************************************************************
File "/usr/share/sage/devel/sage/sage/libs/ppl.pyx", line 1340:
    sage: p.minimize( -x )
Expected:
    {'minimum': False, 'bounded': False, 'inf_d': 0, 'generator': None, 'inf_n': 0}
Got:
    {'minimum': True, 'bounded': False, 'inf_d': 0, 'generator': None, 'inf_n': 0}

On x86 but not on amd64. I am guessing from the configuration log on my amd64 machine and the configure.ac file that --enable-fpmath=no is the same as the default and enable use of 387 and sse instructions. --enable-coefficients also currently default to mpz, and we enable c++ by default. So it seems all good configuration wise.

comment:47 Changed 9 years ago by fbissey

more upsetting. The test pass on my x86-macos set up. I'll will have to check that's not limited to my own old home computer.

comment:48 Changed 9 years ago by vbraun

387 and SSE instructions are floating point math. Which versions of PPL are you using? Am I right in the assumption that this is only with Gentoo's ppl library and not with my ppl spkg? If yes, can you open a separate ticket for Gentoo overlay bugs?

comment:49 Changed 9 years ago by fbissey

ppl is 0.11, same as yours unless you patched it. We don't. For all intent and purpose I am the co-maintainer of sage-on-gentoo by the way. You just asked me to open a bug with myself :) Yes this is not with your ppl spkg, I will have to create a set up to compare results with vanilla sage and see what's different on x86. But just to make my point, we leave enable-fpmath and enable-coefficients to their defaults which happens to be what you choose (unless you can point to something subbtle somewhere else), from configure.ac:

enableval=default
fpmath_may_use_387=yes
fpmath_may_use_sse=yes
AC_MSG_CHECKING([whether to select specific floating point arithmetics])
AC_ARG_ENABLE(fpmath,
  AS_HELP_STRING([--enable-fpmath=INSTRUCTION_SET],
                 [select floating point arithmetics]))
case "${enableval}" in
sse)
  AC_MSG_RESULT(sse)
  OPT_FLAGS="$OPT_FLAGS -msse -mfpmath=sse"
  # The SSE instruction set only supports single precision arithmetics:
  # double and extended precision arithmetics is still done using 387.
  ;;
sse2)
  AC_MSG_RESULT(sse2)
  OPT_FLAGS="$OPT_FLAGS -msse2 -mfpmath=sse"
  # SSE2 still does not support extended precision arithmetics.
  ;;
387)
  AC_MSG_RESULT(387)
  OPT_FLAGS="$OPT_FLAGS -mno-sse -mno-sse2 -mfpmath=387"
  # Note that the -mno-sse* and -mfpmath options are only guaranteed
  # to work with GCC.
  if test x"$GCC" = xyes
  then
    fpmath_may_use_sse=no
  fi
  ;;
sse+387)
  AC_MSG_RESULT(sse+387)
  OPT_FLAGS="$OPT_FLAGS -msse -mfpmath=sse,387"
  ;;
sse2+387)
  AC_MSG_RESULT(sse2+387)
  OPT_FLAGS="$OPT_FLAGS -msse2 -mfpmath=sse,387"
  ;;
default)
  AC_MSG_RESULT(default)
  ;;
no)
  AC_MSG_RESULT(default)
  ;;
*)
  AC_MSG_ERROR([bad value ${enableval} for --enable-fpmath, needs sse, sse2, 387, sse+387, sse2+387, default or no])
  ;;
esac

As you can see just saying "no" leaves you with the same thing as "default" and that is 387 and sse. Same thing for coefficient:

enableval=mpz
AC_MSG_CHECKING([the type of integral values to use as coefficients])
AC_ARG_ENABLE(coefficients,
  AS_HELP_STRING([--enable-coefficients=TYPE],
                 [select the type of the coefficients]))
case "${enableval}" in
native-int8)
  AC_MSG_RESULT([native 8 bits integers])
  coefficient_kind=native
  coefficient_bits=8
  coefficient_mnemonic=nint8
  ;;
native-int16)
  AC_MSG_RESULT([native 16 bits integers])
  coefficient_kind=native
  coefficient_bits=16
  coefficient_mnemonic=nint16
  ;;
native-int32)
  AC_MSG_RESULT([native 32 bits integers])
  coefficient_kind=native
  coefficient_bits=32
  coefficient_mnemonic=nint32
  ;;
native-int64)
  AC_MSG_RESULT([native 64 bits integers])
  coefficient_kind=native
  coefficient_bits=64
  coefficient_mnemonic=nint64
  ;;
checked-int8)
  AC_MSG_RESULT([checked 8 bits integers])
  coefficient_kind=checked
  coefficient_bits=8
  coefficient_mnemonic=int8
  ;;
checked-int16)
  AC_MSG_RESULT([checked 16 bits integers])
  coefficient_kind=checked
  coefficient_bits=16
  coefficient_mnemonic=int16
  ;;
checked-int32)
  AC_MSG_RESULT([checked 32 bits integers])
  coefficient_kind=checked
  coefficient_bits=32
  coefficient_mnemonic=int32
  ;;
checked-int64)
  AC_MSG_RESULT([checked 64 bits integers])
  coefficient_kind=checked
  coefficient_bits=64
  coefficient_mnemonic=int64
  ;;
mpz)
  AC_MSG_RESULT([GMP mpz])
  coefficient_kind=unbounded
  coefficient_bits=0
  coefficient_mnemonic=mpz
  ;;
*)
  AC_MSG_ERROR([bad value ${enableval} for --enable-coefficients, checked-int32, checked-int64, checked-int16, checked-int8, mpz, native-int32, native-int64, native-int16 or native-int8])
  ;;
esac

Again mpz is the default if you don't pass anything.

So unless you have patched these behaviors - and you didn't, we have very similar configurations, we pass

--disable-debugging --disable-optimization --enable-ppl_lpsol --disable-pch --disable-watchdog

Since I have a different result for two different x86 setup I am wondering if the great age of my home machine (close to 8 years) is a factor.

comment:50 follow-up: Changed 9 years ago by vbraun

I know that you are doing the sage-on-gentoo, I'm asking you to open a different ticket on the Sage track for the gentoo ebuild issues and stop hijacking this ticket :-P

As far as I know, I'm not using any floating point code in ppl. So your configuration seems to be fine. But clearly your ppl returns a wrong result. Is your compiler/toolchain up to date on your antique computer?

comment:51 in reply to: ↑ 50 Changed 9 years ago by fbissey

Replying to vbraun:

I know that you are doing the sage-on-gentoo, I'm asking you to open a different ticket on the Sage track for the gentoo ebuild issues and stop hijacking this ticket :-P

As far as I know, I'm not using any floating point code in ppl. So your configuration seems to be fine. But clearly your ppl returns a wrong result. Is your compiler/toolchain up to date on your antique computer?

Yes it is (gcc-4.5.2/binutils-2.20.1/glibc-2.11.2). But I will abide to your requests and stop spamming this ticket. I will only abuse it again to post positive reviews (I hope).

comment:52 Changed 9 years ago by vbraun

I had some sig_on without corresponding sig_off's when exceptions were raised, the updated patch fixes that. Now everything plays nicely with Jeroen' sigon/off debugging patch at #10030.

Marshall, considering that it now builds on t2, how about a positive review? ;-)

comment:53 Changed 9 years ago by fbissey

I unfortunately not give a positive review. I tested this again on a vanilla sage-4.6.2.alpha4 (as in _NOT_ sage-on-gentoo) with the latest version of the patch, spkg, install and deps and I still get the same doctest failures. sage -testall didn't report any broken tests before applying the patch. My opinion is that if this gets merged in an alpha release we will see this failure more on 32bit linux.

comment:54 Changed 9 years ago by vbraun

Can you give more details? Which hardware is it failing on, what is the result of the PPL testsuite (build with SAGE_CHECK=yes), is your gcc configured to use ppl (how is that working out for you), can you reproduce the issue with a supported linux distribution, ...

comment:55 Changed 9 years ago by vbraun

  • Description modified (diff)

I've updated the PPL spkg to the new upstream version 0.11.1 and tested it on Fedora 14 i386. It passes all tests on my system, so I'm pretty confident that it works fine on 32bit.

comment:56 Changed 9 years ago by mhampton

Volker - I still can't get this to work on t2, but I think that's because of my Solaris ignorance. How did you build and test on Solaris - in particular, did you ever change your environment variables somehow?

comment:57 Changed 9 years ago by vbraun

Oops, I accidentally dropped the fix for the newest_version check that is required on t2, see comment:41. Sorry about that.

Updated spkg fixes it and builds on t2.

Marshall: Are you asking about building all of Sage on t2? I don't remember any specific problem there, just source the t2-setup:

. /usr/local/bin/t2-setup
make

If you put stuff into your .profile and it is not picked up you are using a different shell with different initalization files. You can also use a binary distribution, sage-4.6.1-sunos-32bit-5.10-sun4v-SunOS.tar.gz works.

comment:58 Changed 9 years ago by fbissey

I noticed that 0.11.2 is out and from the release notes it sounds like you had some wish granted:

Bugfixes
========

o  Fixed the semantics of the `--disable-fpmath' configure option
   (which is equivalent to `--enable-fpmath=no').  It now disables all
   floating point computations and, consequently, all numerical
   abstractions based on floating point numbers.

o  The PPL no longer overwrites the SIGILL signal handler.

o  Minor documentation fixes.

o  Portability improved.

comment:59 Changed 9 years ago by vbraun

  • Description modified (diff)

I updated my ppl spkg to 0.11.2, see the link in the ticket description. Builds and tests correctly on F14 x86_64, i384, and on t2.math.

comment:60 Changed 9 years ago by mhampton

I'm going to look at this again this week, hopefully can give a positive review soon. I'm going to try to install on some of the skynet machines and do a little more testing. It would be great to get this into sage-4.7.

I just tested on a mac, with your latest package, and I think an error message might have changed. I get the following doctest "failure" (test is expected to fail, just not with this message):

File "/Users/mh/sagestuff/sage-4.7.alpha2/devel/sage-t1/sage/libs/ppl.pyx", line 4124:
    sage: line.divisor()
Expected:
    Traceback (most recent call last):
    ...
    ValueError: Only points and closure points have a divisor.
Got:
    Traceback (most recent call last):
      File "/Users/mh/sagestuff/sage-4.7.alpha2/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/mh/sagestuff/sage-4.7.alpha2/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/mh/sagestuff/sage-4.7.alpha2/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_108[8]>", line 1, in <module>
        line.divisor()###line 4124:
    sage: line.divisor()
      File "ppl.pyx", line 4131, in sage.libs.ppl.Generator.divisor (sage/libs/ppl.cpp:13949)
        mpz_set(c.value, self.thisptr.divisor().get_mpz_t())
    ValueError: PPL::Generator::divisor():
    *this is neither a point nor a closure point.

comment:61 Changed 9 years ago by mhampton

  • Status changed from needs_review to needs_info

comment:62 Changed 9 years ago by vbraun

  • Status changed from needs_info to needs_review

Hi Marshall,

Sage-4.7.alpha2 hasn't been released yet, you are cheating :-P

It seems like Cython-0.14.1 got smarter and now translates the C++ std::invalid_argument exception into a Python ValueError, while the older Cython only threw a nondescript RuntimeError. So I don't have to catch the exception any more, perfect. Attached patch fixes the doctest, I only removed the try/except block in divisor().

For anybody who is still using Sage-4.6.x: You will encounter one doctest failure but that is harmless.

Changed 9 years ago by vbraun

Updated patch

comment:63 Changed 9 years ago by mhampton

OK, this looks good on OS X, linux, and solaris. I built and tested on the skynet machine "mark", and everything seemed fine except for some doctest errors which all seem related to timeout issues - not surprising perhaps since mark is quite slow and has a load of other processes on it right now. So I'm not sure I would consider those failures blockers for this ticket. The failures for mark are at: http://sage.math.washington.edu/home/mhampton/ppltest2.txt

comment:64 Changed 9 years ago by mhampton

  • Status changed from needs_review to positive_review

Sorry, I needed to read the test_executable docs. Clearly these are just timeout issues, and running those doctests with longer values works fine on mark.

I've tested this on a variety of OS X 10.6, linux, and solaris machines now. Hopefully this can be included in an alpha or rc0 of sage-4.7 for wider testing.

comment:65 Changed 9 years ago by novoselt

Can someone please post step-by-step instructions how to apply it? I tried to install the package (it went fine) and apply the last patch, but when I build I get the error

----------------------------------------------------------
sage: Building and installing modified Sage library files.


Installing c_lib
scons: `install' is up to date.
Updating Cython code....
Traceback (most recent call last):
  File "setup.py", line 859, in <module>
    queue = compile_command_list(ext_modules, deps)
  File "setup.py", line 819, in compile_command_list
    dep_file, dep_time = deps.newest_dep(f)
  File "setup.py", line 726, in newest_dep
    for f in self.all_deps(filename):
  File "setup.py", line 707, in all_deps
    for f in self.immediate_deps(filename):
  File "setup.py", line 689, in immediate_deps
    self._deps[filename] = self.parse_deps(filename)
  File "setup.py", line 679, in parse_deps
    raise IOError, "could not find dependency %s included in %s."%(path, filename)
IOError: could not find dependency ppl.hh included in sage/libs/ppl.pyx.
sage: There was an error installing modified sage library code.

comment:66 Changed 9 years ago by mhampton

Andrey - that should have worked. Installing the package should put ppl.hh in $SAGE_ROOT/local/include/ - is it there? If the spkg installed OK, I don't know what would be wrong except the error that I sometimes make of working on two different copies of sage.

comment:67 Changed 9 years ago by novoselt

It is, but I have realized that I didn't install dependencies. I guess I'll install 4.7.alpha1 and try again.

comment:68 Changed 9 years ago by novoselt

  • Milestone changed from sage-feature to sage-4.7

Everything applies smooth now, sorry for the false alarm!

comment:69 Changed 9 years ago by jdemeyer

  • Description modified (diff)

Some administrative issues:

  1. Now that #9433 is merged, changes to spkg/install and spkg/standard/deps should be made using hg export tip (producing an actual HG changeset for the sage_root repository).
  2. Please state clearly which patches have to be applied where.

comment:70 Changed 9 years ago by jdemeyer

  • Status changed from positive_review to needs_work

Changed 9 years ago by vbraun

Patch to SAGE_ROOT repo

Changed 9 years ago by vbraun

patch to SAGE_LOCAL/bin repo

comment:71 Changed 9 years ago by vbraun

  • Description modified (diff)

The patches are explained in the ticket description.

comment:72 Changed 9 years ago by vbraun

  • Status changed from needs_work to positive_review

back to positive review since I only repackaged existing changes.

comment:73 Changed 9 years ago by jdemeyer

This ticket will also be an interesting test for #9433: adding a new spkg has not been tested before.

comment:74 Changed 9 years ago by jdemeyer

  • Merged in set to sage-4.7.alpha3
  • Resolution set to fixed
  • Status changed from positive_review to closed

Changed 9 years ago by jdemeyer

Patch to MANIFEST.in

comment:75 Changed 9 years ago by jdemeyer

  • Description modified (diff)
  • Status changed from closed to needs_work

comment:76 Changed 9 years ago by jdemeyer

  • Status changed from needs_work to needs_review

comment:77 Changed 9 years ago by jdemeyer

  • Status changed from needs_review to needs_work

On OS X 10.6 (both 32-bit and 64-bit):

sage -t -long  -force_lib devel/sage/sage/libs/ppl.pyx
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 4178:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_110[8]>", line 1, in <module>
        (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest###line 4178:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/lib/python/site-packages/sage/tests/cmdline.py", line 366, in test_executable
        raise RuntimeError("timeout in test_executable()")
    RuntimeError: timeout in test_executable()
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 4179:
    sage: print err
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_110[9]>", line 1, in <module>
        print err###line 4179:
    sage: print err
    NameError: name 'err' is not defined
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 4444:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_124[8]>", line 1, in <module>
        (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest###line 4444:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/lib/python/site-packages/sage/tests/cmdline.py", line 366, in test_executable
        raise RuntimeError("timeout in test_executable()")
    RuntimeError: timeout in test_executable()
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 4445:
    sage: print err
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_124[9]>", line 1, in <module>
        print err###line 4445:
    sage: print err
    NameError: name 'err' is not defined
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5056:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_149[8]>", line 1, in <module>
        (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest###line 5056:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/lib/python/site-packages/sage/tests/cmdline.py", line 366, in test_executable
        raise RuntimeError("timeout in test_executable()")
    RuntimeError: timeout in test_executable()
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5057:
    sage: print err
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_149[9]>", line 1, in <module>
        print err###line 5057:
    sage: print err
    NameError: name 'err' is not defined
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5293:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_161[8]>", line 1, in <module>
        (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest###line 5293:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/lib/python/site-packages/sage/tests/cmdline.py", line 366, in test_executable
        raise RuntimeError("timeout in test_executable()")
    RuntimeError: timeout in test_executable()
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5294:
    sage: print err
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_161[9]>", line 1, in <module>
        print err###line 5294:
    sage: print err
    NameError: name 'err' is not defined
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5572:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_176[5]>", line 1, in <module>
        (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest###line 5572:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/lib/python/site-packages/sage/tests/cmdline.py", line 366, in test_executable
        raise RuntimeError("timeout in test_executable()")
    RuntimeError: timeout in test_executable()
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5573:
    sage: print err
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_176[6]>", line 1, in <module>
        print err###line 5573:
    sage: print err
    NameError: name 'err' is not defined
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5822:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_189[5]>", line 1, in <module>
        (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest###line 5822:
    sage: (out, err, ret) = test_executable(['sage', '-c', sage_cmd]);   # indirect doctest
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/lib/python/site-packages/sage/tests/cmdline.py", line 366, in test_executable
        raise RuntimeError("timeout in test_executable()")
    RuntimeError: timeout in test_executable()
**********************************************************************
File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 5823:
    sage: print err
Exception raised:
    Traceback (most recent call last):
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/Users/buildbot/build/sage/bsd-1/bsd_full/build/sage-4.7.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_189[6]>", line 1, in <module>
        print err###line 5823:
    sage: print err
    NameError: name 'err' is not defined
**********************************************************************

Note the default timeout of test_executable() is 10 seconds, which might simply be too short.

comment:78 Changed 9 years ago by jdemeyer

  • Merged in sage-4.7.alpha3 deleted

On RHEL 5.3-64 (cleo), SUSE ES10-64 (iras), OpenSolaris? 06.2009-32 (hawk), SunOS 5.10-32 (t2):

File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 1413:
    sage: p.minimize( -x )
Expected:
    {'minimum': False, 'bounded': False, 'inf_d': 0, 'generator': None, 'inf_n': 0}
Got:
    {'minimum': True, 'bounded': False, 'inf_d': 0, 'generator': None, 'inf_n': 0}

comment:79 Changed 9 years ago by jdemeyer

On SunOS 5.10-32 (t2), OpenSolaris? 06.2009-32 (hawk):

File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.alpha3/devel/sage-main/sage/libs/ppl.pyx", line 1342:
    sage: p.maximize( +x )
Expected:
    {'sup_d': 0, 'sup_n': 0, 'bounded': False, 'maximum': False, 'generator': None}
Got:
    {'sup_d': 0, 'sup_n': 0, 'bounded': False, 'maximum': True, 'generator': None}

comment:80 Changed 9 years ago by jdemeyer

See #11100 for the timeout problem.

comment:81 Changed 9 years ago by vbraun

  • Description modified (diff)
  • Status changed from needs_work to needs_review

I finally got the problem with the maximize/minimize doctest failure. If the linear function is not bounded on the polyhedron then it doesn't make sense to ask for whether its a maximum or minimum. And PPL then does not return a consistent value for the corresponding field.

In the attached patch I changed return value of the maximize/minimize methods to only return {'bounded':False} if the linear program is unbounded, since all other dictionary entries are not well-defined in that case. If the linear program is bounded then the returned dictionary is the same as before and contains information about the sup/inf value, whether it is a maximum, and where it is attained.

The new patch trac_10039_ppl_fix_extremize.patch needs to be applied on top of trac_10039_parma_polyhedra_library.patch, see also the ticket description. The new patch needs review.

comment:82 Changed 9 years ago by fbissey

That's good news and that wasn't just me then! I have my hands full at the moment so I won't be able to review it quickly but I am sure you have another reviewer around :)

comment:83 Changed 9 years ago by jdemeyer

  • Reviewers changed from Marshall Hampton to Marshall Hampton, Jeroen Demeyer
  • Status changed from needs_review to positive_review

New patch looks good to me. Still needs to be tested on the buildbots, but positive review in the hope that everything works fine.

comment:84 Changed 9 years ago by jdemeyer

  • Status changed from positive_review to needs_work

Volker, you just need to put a proper commit message on your patch...

Changed 9 years ago by vbraun

Updated patch

Changed 9 years ago by vbraun

Updated patch

comment:85 Changed 9 years ago by vbraun

  • Status changed from needs_work to positive_review

Oops, sorry. New trac_10039_ppl_fix_extremize.patch now has a commit message, no other changes. Setting this back to positive review as only metadata changed.

comment:86 Changed 9 years ago by jdemeyer

  • Merged in set to sage-4.7.alpha4
  • Status changed from positive_review to closed

comment:87 Changed 9 years ago by jdemeyer

  • Description modified (diff)

www.stp.dias.ie seems to be down, so I'm mirroring the spkg myself.

Note: See TracTickets for help on using tickets.