Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#21094 closed enhancement (fixed)

Sage package for SCIP integer programming solver

Reported by: mkoeppe Owned by:
Priority: major Milestone: sage-7.3
Component: packages: experimental Keywords:
Cc: malb, schilly, jason, dimpase, vdelecroix, chapoton, jdemeyer, slelievre, tscrim, embray, vbraun Merged in:
Authors: Martin Albrecht, Matthias Koeppe Reviewers: Travis Scrimshaw, Jeroen Demeyer
Report Upstream: N/A Work issues:
Branch: 1c070e4 (Commits) Commit:
Dependencies: #21151 Stopgaps:

Description (last modified by mkoeppe)

Split out from the larger ticket #10879. This ticket just adds an spkg, not the Cython interface.

"SCIP is currently one of the fastest non-commercial mixed integer programming (MIP) solvers. It is also a framework for Constraint Integer Programming and branch-cut-and-price. It allows total control of the solution process and the access of detailed information down to the guts of the solver." -- http://scip.zib.de/

Features interesting to Sage:

  • SCIP is pretty fast for Mixed Integer Programming
  • SCIP is a Constraint Integer Programming solver and allows non-linear constraints
  • SCIP's source code is available
  • SoPlex? (the LP solver which is part of scipoptsuite) has an exact mode which is said to be working well
  • A beta version of SCIP can do exact MIP: http://scip.zib.de/#exact using qsopt_ex (see also #18766)

However, the SCIP license prevents us from redistributing the SCIP source code: http://scip.zib.de/#license

To build a SCIP for Sage do:

Upstream discussion: http://listserv.zib.de/pipermail/scip/2016-July/002853.html

Follow-up: #21003: Add SCIP backend using PySCIPOpt

Change History (34)

comment:1 Changed 3 years ago by mkoeppe

  • Branch set to u/mkoeppe/sage_package_for_scip_integer_programming_solver

comment:2 Changed 3 years ago by git

  • Commit set to e21345247bf2d9196ad455fe829e78858cb637e6

Branch pushed to git repo; I updated commit sha1. New commits:

e213452Update build instructions

comment:3 Changed 3 years ago by git

  • Commit changed from e21345247bf2d9196ad455fe829e78858cb637e6 to 0b4219c7ea4570a411116b665883afc20d4e44d0

Branch pushed to git repo; I updated commit sha1. New commits:

55307b3build soplex with -install_name
0b4219cFix library dependencies on Mac OS X

comment:4 Changed 3 years ago by mkoeppe

  • Cc tscrim added
  • Description modified (diff)
  • Status changed from new to needs_review

It finally builds on Mac OS X. Needs testing / review.

comment:5 Changed 3 years ago by mkoeppe

  • Authors set to Martin Albrecht, Matthias Koeppe

comment:6 Changed 3 years ago by git

  • Commit changed from 0b4219c7ea4570a411116b665883afc20d4e44d0 to 9b7a26c0b7fe5f5e77eb445e8b55fe02b6e442d9

Branch pushed to git repo; I updated commit sha1. New commits:

0836b16Unpack soplex and scip where the scipoptsuite Makefile would
29a8853Use OPT=dbg build to fix segfault on Mac OS X
9b7a26cspkg-install: check scip starts, spkg-check: run testsuite

comment:7 Changed 3 years ago by git

  • Commit changed from 9b7a26c0b7fe5f5e77eb445e8b55fe02b6e442d9 to 5d900b018ff4aa17e8d133e222941506309cee80

Branch pushed to git repo; I updated commit sha1. New commits:

5d900b0Build scip linking against libscip.

comment:8 Changed 3 years ago by tscrim

  • Status changed from needs_review to needs_work

On 7.3.rc0 with my Ubuntu 14.04 LTS, I could not get this to build:

Found local metadata for scipoptsuite-3.2.1
Using cached file /home/travis/sage-build/upstream/scipoptsuite-3.2.1.tgz
scipoptsuite-3.2.1
====================================================
Setting up build directory for scipoptsuite-3.2.1
Finished extraction
****************************************************
Host system:
Linux apricot 3.13.0-87-generic #133-Ubuntu SMP Tue May 24 18:32:09 UTC 2016 x86_64 x86_6
4 x86_64 GNU/Linux
****************************************************
C compiler: gcc
C compiler version:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.9/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.9.3-8ubuntu2~14.04' --wi
th-bugurl=file:///usr/share/doc/gcc-4.9/README.Bugs --enable-languages=c,c++,java,go,d,fo
rtran,objc,obj-c++ --prefix=/usr --program-suffix=-4.9 --enable-shared --enable-linker-bu
ild-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx
-include-dir=/usr/include/c++/4.9 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enabl
e-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-ob
ject --disable-vtable-verify --enable-plugin --with-system-zlib --disable-browser-plugin 
--enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.9
-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.9-amd64 -
-with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.9-amd64 --with-arch-directory=amd
64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --d
isable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enab
le-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host
=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.9.3 (Ubuntu 4.9.3-8ubuntu2~14.04) 
****************************************************
soplex-2.2.1/COPYING
soplex-2.2.1/INSTALL
...
soplex-2.2.1/src/usertimer.cpp
soplex-2.2.1/src/wallclocktimer.cpp
patching file Makefile
Compiling SoPlex
make: Entering an unknown directory
make: *** empty string invalid as file name.  Stop.
make: Leaving an unknown directory
Error building SOPLEX.

The log file doesn't contain anything else of interest, but I can post it if you want. Tell me what you need from me or what I can do to debug.

comment:9 Changed 3 years ago by mkoeppe

"Entering an unknown directory", very mysterious. Perhaps you can go to the temporary build directory and do a "bash -x spkg-install" and post the result. Thanks!

comment:10 Changed 3 years ago by git

  • Commit changed from 5d900b018ff4aa17e8d133e222941506309cee80 to a050604a747f0c4750b3a9ff216a56d0895f2896

Branch pushed to git repo; I updated commit sha1. New commits:

a050604Install libscip also as libscipopt

comment:11 follow-up: Changed 3 years ago by git

  • Commit changed from a050604a747f0c4750b3a9ff216a56d0895f2896 to c21e1bb613acbad9a76957ce8bbe48ef1bb8bef5

Branch pushed to git repo; I updated commit sha1. New commits:

c21e1bbRedo SCIP patches, install missing headers

comment:12 Changed 3 years ago by git

  • Commit changed from c21e1bb613acbad9a76957ce8bbe48ef1bb8bef5 to 438a190a939739dfdee1100f63c98243c7176b95

Branch pushed to git repo; I updated commit sha1. New commits:

438a190More fixes for Darwin/non-Darwin

comment:13 Changed 3 years ago by mkoeppe

OK, I've solved this particular problem, but it needs more work regarding the shared libraries on Linux.

comment:14 Changed 3 years ago by tscrim

  • Reviewers set to Travis Scrimshaw

Sorry for the delay in getting back to this. Yes, this installed for me now. However, the spkg-install is non-trivial, so I would prefer someone with a bit more expertise, such as Jeroen (jdemeyer), Erik (embray) or Volker (vbraun), to look over it before giving this a positive review.

comment:15 Changed 3 years ago by git

  • Commit changed from 438a190a939739dfdee1100f63c98243c7176b95 to 7075b83aa81328746462bb487f69ac30c4cdd943

Branch pushed to git repo; I updated commit sha1. New commits:

7075b83Pacth SCIP Makefile more to get library dependencies right

comment:16 Changed 3 years ago by git

  • Commit changed from 7075b83aa81328746462bb487f69ac30c4cdd943 to b372675e069d10a4530fec53a161657efb5259e5

Branch pushed to git repo; I updated commit sha1. New commits:

b372675Fixup

comment:17 Changed 3 years ago by git

  • Commit changed from b372675e069d10a4530fec53a161657efb5259e5 to 49659e3e0fb62b30e4231f8b20f1c9492dbc85f6

Branch pushed to git repo; I updated commit sha1. New commits:

49659e3Fixup

comment:18 Changed 3 years ago by mkoeppe

  • Reviewers changed from Travis Scrimshaw to Travis Scrimshaw + NEEDS MORE REVIEWERS
  • Status changed from needs_work to needs_review

OK, now it works also on Linux. Needs review.

comment:19 follow-up: Changed 3 years ago by tscrim

  • Cc embray vbraun added

FYI, recent changes also compile on my Ubuntu.

Erik, Volker, or Jeroen, do one of you have time to look over the build setup? Also, a more general question, can we rehost the tarball?

comment:20 in reply to: ↑ 19 Changed 3 years ago by mkoeppe

Replying to tscrim:

can we rehost the tarball?

No. Users need to download it themselves from the ZIB server. Details: http://listserv.zib.de/pipermail/scip/2015-July/002448.html

comment:21 Changed 3 years ago by mkoeppe

  • Dependencies set to #21151

comment:22 follow-up: Changed 3 years ago by jdemeyer

Given that we cannot host the tarball, I would really want to see this as an "experimental" package, just like the component on this ticket suggests.

Besides this, if the package has successfully been built on Linux and OS X, that is sufficient for me to have positive_review.

comment:23 Changed 3 years ago by tscrim

Matthias, have you asked SCIP whether or not we could host the tarball on our mirrors and/or have Sage have some kind of automatic download of the tarball (but not hosted on our mirrors)?

comment:24 follow-up: Changed 3 years ago by dimpase

We did have a long discussion with ZIB on this, and the answer was no. They want to count their downloads, full stop.

comment:25 Changed 3 years ago by git

  • Commit changed from 49659e3e0fb62b30e4231f8b20f1c9492dbc85f6 to 1c070e44012ab53f5c1074d320f3fe6b9f94a8e2

Branch pushed to git repo; I updated commit sha1. New commits:

527147eMerge tag '7.3.rc0' into t/21094/sage_package_for_scip_integer_programming_solver
1c070e4scipoptsuite: Change package type to experimental

comment:26 in reply to: ↑ 22 Changed 3 years ago by mkoeppe

Replying to jdemeyer:

Given that we cannot host the tarball, I would really want to see this as an "experimental" package, just like the component on this ticket suggests.

OK, done.

Besides this, if the package has successfully been built on Linux and OS X, that is sufficient for me to have positive_review.

OK, could you take a look at #21151, which I have marked as a dependency - it's a minor update to TOPCOM so that it still installs when scipoptsuite has been installed.

comment:27 in reply to: ↑ 24 ; follow-up: Changed 3 years ago by tscrim

Replying to dimpase:

We did have a long discussion with ZIB on this, and the answer was no. They want to count their downloads, full stop.

We can't even set it up so that running sage -i scipoptsuite automatically goes through and does the download?

comment:28 follow-up: Changed 3 years ago by jdemeyer

  • Reviewers changed from Travis Scrimshaw + NEEDS MORE REVIEWERS to Travis Scrimshaw, Jeroen Demeyer
  • Status changed from needs_review to positive_review

I think this is sufficient for an experimental package.

comment:29 in reply to: ↑ 27 Changed 3 years ago by mkoeppe

Replying to tscrim:

Replying to dimpase: We can't even set it up so that running sage -i scipoptsuite automatically goes through and does the download?

As per http://listserv.zib.de/pipermail/scip/2015-July/002448.html :

Q: Users could "download and install" this copy by running "sage -i scip"

A: In this case you would have to ensure that people notice and acknowledge that they use SCIP under the Academic License, which excludes any commercial usage.

So some kind of license display and acknowledgement prompt could be done. I don't want to work on this, though. If you want to add this, please make a follow-up ticket.

comment:30 in reply to: ↑ 28 Changed 3 years ago by mkoeppe

Replying to jdemeyer:

I think this is sufficient for an experimental package.

Thanks!

comment:31 Changed 3 years ago by mkoeppe

comment:32 Changed 3 years ago by vbraun

  • Branch changed from u/mkoeppe/sage_package_for_scip_integer_programming_solver to 1c070e44012ab53f5c1074d320f3fe6b9f94a8e2
  • Resolution set to fixed
  • Status changed from positive_review to closed

comment:33 in reply to: ↑ 11 Changed 3 years ago by jdemeyer

  • Commit 1c070e44012ab53f5c1074d320f3fe6b9f94a8e2 deleted

Replying to git:

Branch pushed to git repo; I updated commit sha1. New commits:

c21e1bbRedo SCIP patches, install missing headers

Why were the patches put inside a scip subdirectory? I see no reason for that (scipoptsuite is the only package doing this)

comment:34 Changed 3 years ago by mkoeppe

SCIPoptsuite itself is a "distribution", consisting of SoPlex?, SCIP, and other packages, whose tar.gz files are included in the tarball of SCIPoptsuite. (Not all of these packages have a separate upstream download.)

The patches are relative to the scip subdirectory that is created after unpacking the included SCIP tar.gz. They are generated from a git repository of SCIP.

Note: See TracTickets for help on using tickets.