Opened 8 months ago

Closed 8 months ago

#28689 closed defect (fixed)

py3: fix doctests with cbc

Reported by: dcoudert Owned by:
Priority: major Milestone: sage-9.0
Component: linear programming Keywords:
Cc: Merged in:
Authors: David Coudert Reviewers: John Palmieri
Report Upstream: N/A Work issues:
Branch: 0bbdca8 (Commits) Commit: 0bbdca88571af80374d26fe6ab021486b66a8ce7
Dependencies: Stopgaps:

Description (last modified by dcoudert)

The following doctests are failing with py3 and cbc, but not with py2. This is due to a different behavior of round called in get_variable_value.

Using --optional=benzene,bliss,build,cbc,csdp,dochtml,dot2tex,gap_packages,gfortran,igraph,libsemigroups,mcqd,mpir,plantri,python_igraph,sage,tdlib
Doctesting 1 file using 4 threads.
sage -t --long src/sage/numerical/backends/coin_backend.pyx
**********************************************************************
File "src/sage/numerical/backends/coin_backend.pyx", line 423, in sage.numerical.backends.coin_backend.CoinBackend.remove_constraint
Failed example:
    p.get_values([x,y])                          # optional - cbc
Expected:
    [0.0, 3.0]
Got:
    [0, 3]
**********************************************************************
File "src/sage/numerical/backends/coin_backend.pyx", line 461, in sage.numerical.backends.coin_backend.CoinBackend.remove_constraints
Failed example:
    p.get_values(x)                              # optional - cbc
Expected:
    2.0...
Got:
    2
**********************************************************************
File "src/sage/numerical/backends/coin_backend.pyx", line 463, in sage.numerical.backends.coin_backend.CoinBackend.remove_constraints
Failed example:
    p.get_values(y)                              # optional - cbc
Expected:
    0.0...
Got:
    0
**********************************************************************
File "src/sage/numerical/backends/coin_backend.pyx", line 468, in sage.numerical.backends.coin_backend.CoinBackend.remove_constraints
Failed example:
    p.get_values([x,y])                          # optional - cbc
Expected:
    [0.0, 3.0]
Got:
    [0, 3]
**********************************************************************
2 items had failures:
   1 of  13 in sage.numerical.backends.coin_backend.CoinBackend.remove_constraint
   3 of  15 in sage.numerical.backends.coin_backend.CoinBackend.remove_constraints
    [353 tests, 4 failures, 0.34 s]

Change History (15)

comment:1 Changed 8 months ago by dcoudert

  • Branch set to public/linear_programming/28689_cbc
  • Commit set to 997e76f250d286b8fef57d2a06240626126b0450
  • Description modified (diff)
  • Status changed from new to needs_review

The proposed solution unifies the behavior with cplex backend.


New commits:

997e76ftrac #28689: fix doctests with cbc

comment:2 Changed 8 months ago by jhpalmieri

When I try to install cbc, I get an error:

mkdir: /Users/palmieri/Desktop/Sage_stuff/git/sage/local/var/tmp/sage/build/cbc-2.9.4.p0/inst/Users/palmieri/Desktop/Sage_stuff/git/sage/local/include/coin: File exists
make[5]: *** [install-includecoinHEADERS] Error 1
make[5]: *** Waiting for unfinished jobs....

I think it's a race condition. Please try building with MAKE="make -j4" (for example) to see if you run into this. For me, this fixes it:

  • build/pkgs/cbc/spkg-install

    diff --git a/build/pkgs/cbc/spkg-install b/build/pkgs/cbc/spkg-install
    index 058c488ea7..d1b6446620 100644
    a b sed -i -e "s/clock\_gettime ()/Grrrrrrrrrrrr\ ()/g" Cbc/configure || \ 
    1515sdh_configure --enable-cbc-parallel --enable-parallel \
    1616              --enable-gnu-packages --enable-static
    1717sdh_make
    18 sdh_make_install
     18sdh_make_install -j1
Last edited 8 months ago by jhpalmieri (previous) (diff)

comment:3 Changed 8 months ago by dcoudert

I don't understand what you are asking me to try, and note that I'm using a 5 years old macbook air with a dual core i7 1,7 GHz CPU...

comment:4 Changed 8 months ago by jhpalmieri

Please set MAKE="make -j2" and MAKEFLAGS="-j2" (since your machine has 2 cores) and try to build cbc. I have these variables set to make sure I get a parallel build, and cbc fails for me; I'm asking if you see the same behavior. (It may be MAKEFLAGS rather than MAKE which is causing the problem, but the Sage installation guide suggests using either, so we should support setting either.)

If you do see this problem, then see if my proposed fix works.

Maybe other reviewers could do the same testing.

comment:5 Changed 8 months ago by dcoudert

I tried with -j2 and it's working (./sage -f cbc). However, with -j4 it's failing.

comment:6 Changed 8 months ago by jhpalmieri

For me, too.

comment:7 Changed 8 months ago by dcoudert

feel free to push required changes here (branch in public)

comment:8 Changed 8 months ago by jhpalmieri

I get a doctest failure:

sage -t --long --warn-long 64.7 src/sage/numerical/backends/gurobi_backend.pyx
**********************************************************************
File "src/sage/numerical/backends/gurobi_backend.pyx", line 13, in sage.numerical.backends.gurobi_backend
Failed example:
    g.feedback_edge_set(value_only = True, constraint_generation = False)
Exception raised:
    Traceback (most recent call last):
      File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 681, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python3.7/site-packages/sage/doctest/forker.py", line 1123, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.numerical.backends.gurobi_backend[1]>", line 1, in <module>
        g.feedback_edge_set(value_only = True, constraint_generation = False)
      File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python3.7/site-packages/sage/graphs/digraph.py", line 1638, in feedback_edge_set
        value_only=True, solver=solver, verbose=verbose)
      File "/Users/palmieri/Desktop/Sage_stuff/git/sage/local/lib/python3.7/site-packages/sage/graphs/digraph.py", line 1727, in feedback_edge_set
        return Integer(round(p.solve(objective_only=True, log=verbose)))
    TypeError: type sage.rings.real_double.RealDoubleElement doesn't define __round__ method
**********************************************************************
1 item had failures:
   1 of   3 in sage.numerical.backends.gurobi_backend
    [10 tests, 1 failure, 0.16 s]

I don't know what this has to do with cbc, but I get it when it's installed, I don't when it's not.

comment:9 Changed 8 months ago by git

  • Commit changed from 997e76f250d286b8fef57d2a06240626126b0450 to 4310696850f66c55876472626c212b117eb4dd20

Branch pushed to git repo; I updated commit sha1. New commits:

4310696trac 28689: do not install cbc in parallel to avoid race condition

comment:10 Changed 8 months ago by git

  • Commit changed from 4310696850f66c55876472626c212b117eb4dd20 to 0bbdca88571af80374d26fe6ab021486b66a8ce7

Branch pushed to git repo; I updated commit sha1. New commits:

323daa9trac #28689: Merged with 9.0.beta4
0bbdca8trac #28689: fix type issue with get_objective_value

comment:11 Changed 8 months ago by dcoudert

I don't have gurobi installed, but I have the same issue with coin (with and without this patch).

sage: G = digraphs.DeBruijn(2, 3)
sage: G.feedback_edge_set(value_only = True, constraint_generation = False, solver='coin')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-3-1312c6b79adf> in <module>()
----> 1 G.feedback_edge_set(value_only = True, constraint_generation = False, solver='coin')

/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/digraph.py in feedback_edge_set(self, constraint_generation, value_only, solver, verbose)
   1621             D.allow_loops(False)
   1622             FAS = D.feedback_edge_set(constraint_generation=constraint_generation,
-> 1623                                           value_only=value_only, solver=solver, verbose=verbose)
   1624             if value_only:
   1625                 return FAS + self.number_of_loops()

/Users/dcoudert/sage3/sage/local/lib/python3.7/site-packages/sage/graphs/digraph.py in feedback_edge_set(self, constraint_generation, value_only, solver, verbose)
   1725 
   1726             if value_only:
-> 1727                 return Integer(round(p.solve(objective_only=True, log=verbose)))
   1728             else:
   1729                 p.solve(log=verbose)

TypeError: type sage.rings.real_double.RealDoubleElement doesn't define __round__ method
sage: G.feedback_edge_set(value_only = True, constraint_generation = False, solver='glpk')
6
sage: G.feedback_edge_set(value_only = True, constraint_generation = False, solver='cplex')
6

Applying the same receipe than for cplex and gurobi in method get_objective_value fix the issue for me.

comment:12 Changed 8 months ago by jhpalmieri

I don't have gurobi installed either, by the way.

comment:13 Changed 8 months ago by dcoudert

The corresponding doctest in gurobi_backend.pyx is on purpose not restricted to a particular backend. So when you installed cbc / coin you highlighted an issue with this solver (it was your default). I have not seen it as I have cplex as default solver, and the issue has been fixed for cplex and gurobi in #27773. It should now be fixed for all solvers.

comment:14 Changed 8 months ago by jhpalmieri

  • Reviewers set to John Palmieri
  • Status changed from needs_review to positive_review

Okay, looks good to me.

comment:15 Changed 8 months ago by vbraun

  • Branch changed from public/linear_programming/28689_cbc to 0bbdca88571af80374d26fe6ab021486b66a8ce7
  • Resolution set to fixed
  • Status changed from positive_review to closed
Note: See TracTickets for help on using tickets.