# Ticket #9836: trac_9836.patch

File trac_9836.patch, 25.0 KB (added by ncohen, 9 years ago)
• ## doc/en/constructions/index.rst

# HG changeset patch
# User Nathann Cohen <nathann.cohen@gmail.com>
# Date 1283600443 -7200
# Node ID 69e2253c2a5eef5c8f1ccc6e2f52a16f26a52dbd
# Parent  5a26560c3ceded43f0115375e8a8b97de0a253b1
trac 9836 -- creates a linear_programming tutorial and removes the old one from the construction document.

diff -r 5a26560c3ced -r 69e2253c2a5e doc/en/constructions/index.rst
 a linear_algebra linear_codes graph_theory linear_programming rep_theory rings polynomials
• ## deleted file doc/en/constructions/linear_programming.rst

diff -r 5a26560c3ced -r 69e2253c2a5e doc/en/constructions/linear_programming.rst
 + Linear Programming ================== Basics ------ What is a linear program? """"""""""""""""""""""""" A linear program consists of the following two pieces of information: * A linear function, called the objective function, which is to be maximized or minimized, e.g. 2 x + y. * Linear constraints on the variables, e.g. 3 x + y \leq 2 and 2 x + 3 y \leq 8. A linear program solver would then try to find a solution to the system of constraints such that the objective function is optimized, and return specific values for the variables. What is a mixed integer linear program? """"""""""""""""""""""""""""""""""""""" A mixed integer linear program is a linear program such that some variables are forced to take integer values instead of real values. This difference affects the time required to solve a particular linear program. Indeed, solving a linear program can be done in polynomial time while solving a general mixed integer linear program is usually NP-complete, i.e. it can take exponential time, according to a widely-held belief that P \neq NP. Why is linear programming so useful? """""""""""""""""""""""""""""""""""" Linear programming is very useful in many optimization and graph-theoretic problems because of its wide range of expression. A linear program can be written to solve a problem whose solution could be obtained within reasonable time using the wealth of heuristics already contained in linear program solvers. It is often difficult to theoretically determine the execution time of a linear program, though it could produce very interesting results in practice. For more information, consult the Wikipedia page dedicated to linear programming: http://en.wikipedia.org/wiki/Linear_programming How can I solve a linear program using Sage? -------------------------------------------- Sage can solve linear programs or mixed integer linear programs through the class MixedIntegerLinearProgram defined in sage.numerical.mip. To illustrate how it can be used, we will try to solve the following problem: .. MATH:: \text{Maximize: }  & 2 x_1 + x_2 \\ \text{Such that: } & 3 x_1 + 4 x_2 \leq 2.5 \\ & 0.5 \leq 1.2 x_1 + 0.5 x_2 \leq 4 First, we need to discuss MIPVariable and how to read the optimal values when the solver has finished its job. Variables in MixedIntegerLinearProgram """""""""""""""""""""""""""""""""""""""""" A variable linked to an instance of MixedIntegerLinearProgram behaves exactly as a dictionary would. It is declared as follows:: sage: p = MixedIntegerLinearProgram() sage: variable = p.new_variable() The variable variable can contain as many keys as you like, where each key must be unique. For example, the following constraint (where P denotes pressure and T temperature) .. MATH:: 2 T_{\text{Madrid}} + 3 T_{\text{London}} - P_{\text{Seattle}} + \text{flow}_{3, 5} + 8 \text{cost}_{(1, 3)} + x_3 < 5 can be written as:: sage: p = MixedIntegerLinearProgram() sage: temperature = p.new_variable() sage: pressure = p.new_variable() sage: x = p.new_variable() sage: cost = p.new_variable() sage: flow = p.new_variable(dim=2) sage: p.add_constraint(2*temperature["Madrid"] + 3*temperature["London"] - pressure["Seattle"] + flow[3][5] + 8*cost[(1, 3)] + x[3], max=5) This example shows different possibilities for using the MixedIntegerLinearProgram class. You would not need to declare so many variables in some common applications of linear programming. Notice how the variable flow is defined: you can use any hashable object as a key for a MIPVariable, but if you think you need more than one dimension, you need to explicitly specify it when calling MixedIntegerLinearProgram.new_variable(). For the user's convenience, there is a default variable attached to a linear program. The above code listing means that each variable actually represents a property of a set of objects (these objects are strings in the case of temperature or a pair in the case of cost). In some cases, it is useful to define an absolute variable which will not be indexed on anything. This can be done as follows:: sage: p = MixedIntegerLinearProgram() sage: B = p.new_variable() sage: p.set_objective(p["first unique variable"] + B[2] + p[-3]) In this case, two of these "unique" variables are defined through p["first unique variable"] and p[-3]. Let's solve this system """"""""""""""""""""""" Now that we know what variables are, we are only several steps away from solving our system:: sage: # First, we define our MixedIntegerLinearProgram object, sage: # setting maximization=True. sage: p = MixedIntegerLinearProgram(maximization=True) sage: x = p.new_variable() sage: # Definition of the objective function sage: p.set_objective(2*x[1] + x[2]) sage: # Next, the two constraints sage: p.add_constraint(3*x[1] + 4*x[2], max=2.5) sage: p.add_constraint(1.5*x[1]+0.5*x[2], max=4, min=0.5) sage: p.solve()                # optional - requires GLPK or COIN-OR/CBC 1.6666666666666667 sage: x_sol = p.get_values(x)  # optional - requires GLPK or COIN-OR/CBC sage: print x_sol              # optional - requires GLPK or COIN-OR/CBC {1: 0.83333333333333337, 2: 0.0} The value returned by MixedIntegerLinearProgram.solve() is the optimal value of the objective function. To read the values taken by the variables, one needs to call the method MixedIntegerLinearProgram.get_values() which can return multiple values at the same time if needed (type MixedIntegerLinearProgram.get_values? for more information on this function). Some famous examples -------------------- Vertex cover in a graph """"""""""""""""""""""" Let G = (V, E) be a graph with vertex set V and edge set E. In the vertex cover problem, we are given G and we want to find a subset S \subseteq V of minimal cardinality such that each edge e is incident to at least one vertex in S. In order to achieve this, we define a binary variable b_v for each vertex v. The vertex cover problem can be expressed as the following linear program: .. MATH:: \text{Maximize: }  & \sum_{v \in V} b_v \\ \text{Such that: } & \forall (u, v) \in E, b_u + b_v \geq 1 \\ & \forall v, b_v \text{ is a binary variable} In the linear program, the syntax is exactly the same:: sage: g = graphs.PetersenGraph() sage: p = MixedIntegerLinearProgram(maximization=False) sage: b = p.new_variable() sage: for u, v in g.edges(labels=None): ...       p.add_constraint(b[u] + b[v], min=1) sage: p.set_binary(b) And you need to type p.solve() to see the result. Maximum matching in a graph """"""""""""""""""""""""""" In the maximum matching problem, we are given a graph G = (V, E) and we want a set of edges M \subseteq E of maximum cardinality such that no two edges from M are adjacent: .. MATH:: \text{Maximize: }  & \sum_{e \in E} b_e \\ \text{Such that: } & \forall v \in V, \sum_{(v,u) \in E} b_{vu} \leq 1 \\ & \forall e \in E, b_e \text{ is a binary variable} Here, we use Sage to solve the maximum matching problem for the case of the Petersen graph:: sage: g = graphs.PetersenGraph() sage: p = MixedIntegerLinearProgram() sage: b = p.new_variable(dim=2) sage: for u in g.vertices(): ...       p.add_constraint(sum([b[u][v] for v in g.neighbors(u)]), max=1) sage: for u, v in g.edges(labels=None): ...       p.add_constraint(b[u][v] + b[v][u], min=1, max=1) And the next step is p.solve(). Solvers ------- Sage solves linear programs by calling specific libraries. The following libraries are currently supported : * CBC _: A solver from from COIN-OR _ (CPL -- Free) * CPLEX _: A solver from ILOG _ (Proprietary) * GLPK _: A solver from GNU _ (GPL3, Free) Installing GLPK or CBC """""""""""""""""""""""" GPLK and CBC being free softwares, they can be easily installed as follows:: sage: # To install GLPK sage: install_package("glpk")  # not tested sage: # To install COIN-OR Branch and Cut (CBC) sage: install_package("cbc")   # not tested Installing CPLEX """""""""""""""" ILOG CPLEX, on the other hand, is Proprietary -- to use it through Sage, you must first be in possession of : * A valid license file * A compiled version of the CPLEX library (usually named libcplex.a) * The header file cplex.h The license file path must be set the value of the environment variable ILOG_LICENSE_FILE. For example, you can write :: export ILOG_LICENSE_FILE=/path/to/the/license/ilog/ilm/access_1.ilm at the end of your .bashrc file. As Sage also needs the files libcplex.a and cplex.h, the easiest way is to create symbolic links toward these files in the appropriate directories : * libcplex.a -- in SAGE_ROOT/local/lib/, type :: ln -s /path/to/lib/libcplex.a . * cplex.h -- in SAGE_ROOT/local/include/, type :: ln -s /path/to/include/cplex.h . Once this is done, and as CPLEX is used in Sage through the Osi library, which is part of the Cbc package, you can type:: sage: install_package("cbc")  # not tested or, if you had already installed Cbc :: sage: install_package("cbc", force = True)  # not tested to reinstall it.
• ## doc/en/thematic_tutorials/index.rst

diff -r 5a26560c3ced -r 69e2253c2a5e doc/en/thematic_tutorials/index.rst
 a functional_programming group_theory linear_programming Indices and tables ==================
• ## new file doc/en/thematic_tutorials/linear_programming.rst

diff -r 5a26560c3ced -r 69e2253c2a5e doc/en/thematic_tutorials/linear_programming.rst
 - (Mixed Integer) Linear Programming ================================== This document explains the use of Linear Programming (LP) -- and of Mixed Integer Linear Programming (MILP) -- in Sage by illustrating it with several problems it can solve. Most of the examples given are motivated by graph-theoretic concerns, and should be understandabe without any specific knowledge of this field. As a tool in Combinatorics, using Linear Programming amounts to understanding how to reformulate an optimization (or existence) problem through linear constraints. .. centered:: *(this is a translation of a chapter from the book "Calcul mathematique avec Sage")* Definition ---------- Set to bear once more upon our shoulders the heavy weight of mathematical formalism, let us give here the usual definition of what a Linear Program is : it is defined by a matrix A: \mathbb{R}^m\mapsto \mathbb{R}^n, along with two vectors b,c\in \mathbb{R}^n. Solving a Linear Program is a quest for a vector x maximizing an *objective* function and satisfying a set of constraints, i.e. .. MATH:: c^t x = \max_{x' \text{ such that}Ax'\leq b}c^t x' (where the ordering u\leq u' between two vectors means that the entries of u' are pairwise greater than the entries of u). We also write : .. MATH:: \text{Max : }&c^t x\\ \text{ Such that : }&Ax\leq b Equivalently, we can also say that solving a linear program amounts to maximizing a linear function defined over a polytope (preimage or A^{-1}(\leq b)). These definitions, however, do not tell us how to use linear programming in combinatorics. In the following, we will try to fix this by showing how to solve optimization problems like the Knapsack problem, the Maximum Matching problem, and a Flow problem. Mixed Integer Linear Programming -------------------------------- There is a bad news coming along with this definition of linear programming : a LP can be solved in polynomial time. This is indeed a bad news, because this would mean than unless we define LP of exponential size, we can not expect LP to solve NP-complete problems, which would be a disappointment. On a brighter side, it becomes NP-Complete to solve a Linear Program  if we are allowed to specify constraints of a different kind : requiring that some variables be integers instead of real values. Such a LP is actually called a "Mixed Integer Linear Program" (some variables can be integers, some other reals). Hence, we can expect to find in the MILP framework a *wide* range of expressivity. Practically ----------- The MILP class ^^^^^^^^^^^^^^^^^^ The MILP class in Sage represents a ... MILP ! It is also used to solve regular LP. It has a very small number of methods, meant to define our set of constraints and variables, then to read the solution found by the solvers once computed. It is also possible to export a MILP defined with Sage to a .lp or .mps file, understood by most solvers. Let us ask Sage to solve the following LP : .. MATH:: \text{Max : }&x+y-3z\\ \text{Such that : }&x+2y \leq 4\\ \text{}&5z  - y \leq 8\\ To achieve it, we need to define a corresponding MILP object, along with the 3 variables we need :: sage: p = MixedIntegerLinearProgram() sage: x, y, z = p['x'], p['y'], p['z'] The objective function .. link :: sage: p.set_objective( x + y + 3*z ) And finally the constraints .. link :: sage: p.add_constraint( x + 2*y <= 4 ) sage: p.add_constraint( 5*z - y  <= 8 ) The solve method returns by default the optimal value reached by the objective function .. link :: sage: round(p.solve(),2) 8.8 We can read the optimal assignation found by the solver for x, y and z through the get_values method .. link :: sage: round(p.get_values(x),2) 4.0 sage: round(p.get_values(y), 2) 0.0 sage: round(p.get_values(z), 2) 1.6 Variables ^^^^^^^^^ The variables associated with an instance of MILP belong to the MIPVariable class, though we should not be concerned with this. In the previous example, we obtained these variables through the "shortcut" p['x'], which is easy enough when our LP is defined over a small number of variables. This being said, the LP/MILP we will present afterwards very often require one to associate one -- or many -- variables to each member of a list of objects, which can be integers, or the vertices or edges of a graph, among plenty of other alternatives. This means we will very soon need to talk about vectors of variables, if not of dictionaries of variables. If a LP requires us to define variables named x_1, \dots, x_{15}, we will this time make use of the new_variable method .. link :: sage: x = p.new_variable() It is now very easy to define constraints using our 15 variables .. link :: sage: p.add_constraint( x[1] + x[12] - x[14] >= 8 ) Notice that we did not need to define the length of our vector. Actually, x would accept any immutable object as a key, as a dictionary would. We can now write .. link :: sage: p.add_constraint( x["I am a valid key"] ...                   + x[("a",pi)] <= 3 ) Other LP may require variables indexed several times. Of course, it is already possible to emulate it by using tuples like x[(2,3)], though to keep the code understandable the method new_variable accepts as a parameter the integer dim, which lets us define the dimension of the variable. We can now write .. link :: sage: y = p.new_variable(dim = 2) sage: p.add_constraint( y[3][2] + x[5] == 6) Typed variables """"""""""""""" By default, all the LP variables are assumed to be non-negativereals. They can be defined as binary through the parameter binary = True (or integer with integer = True). Lower and upper bounds can be defined or re-defined (for instance when you want some variables to be negative) using the methods set_min and set_max. It is also possible to change the type of a variable after it has been created with the methods set_binary and set_integer. Basic Linear Programs --------------------- Knapsack ^^^^^^^^ The *Knapsack* problem is the following : given a collection of items having both a weight and a *usefulness*, we would like to fill a bag whose capacity is constrained through maximizing the usefulness of the items it contains (we will here consider their sum). To achieve this, we have to associate to each object o of our collection C a binary variable taken[o], set to 1 when the object is in the bag, and to 0 otherwise. We are trying to solve the following MILP .. MATH:: \text{Max : }&\sum_{o\in L}usefulness_o\times taken_o\\ \text{Tel que : }&\sum_{o\in L} poids_o\times taken_o \leq C\\ Using \Sage, we will give to our items a random weight :: sage: C = 1 .. link :: sage: L = ["Casserole", "Livre", "Couteau", ...        "Gourde", "Lampe de poche"] .. link :: sage: L.extend( ["divers_"+str(i) for i in range(20)] ) .. link :: sage: poids = {} sage: usefulness = {} .. link :: sage: set_random_seed(685474) sage: for o in L: ...      poids[o]   = random() ...      usefulness[o] = random() We can now define the MILP itself .. link :: sage: p = MixedIntegerLinearProgram() sage: taken = p.new_variable( binary = True ) .. link :: sage: p.add_constraint( ...     sum( poids[o] * taken[o] for o in L ) <= C ) .. link :: sage: p.set_objective( ...     sum( usefulness[o] * taken[o] for o in L ) ) .. link :: sage: p.solve() 3.1502766806530307 sage: taken = p.get_values(taken) The solution found is (of course) admissible .. link :: sage: sum( poids[o] * taken[o] for o in L ) 0.69649597966191712 Should we take a flashlight ? .. link :: sage: taken["Lampe de poche"] 1.0 Wise advice. Based on purely random considerations. Matching -------- Given a graph G, a matching is a set of pairwise disjoint edges. The empty set being a matching, we naturally focus our attention on maximum matchings : we want to find in a graph a matching whose cardinality is maximal. Computing the maximum matching in a graph is a polynomial problem, which is a famous result of Edmonds : his algorithm is based on local improvements, and the proof that a given matching is maximum if it can not be improved. This algorithm is not the hardest to implement among those Graph Theory can offer, though this problem can be modeled with a very simple MILP. To do it, we need -- as previously -- to associate a binary variable to each one of our objects : the edges of our graph (a value of 1 meaning that the corresponding edge is included in the maximum matching). Our constraint on the edges taken being that they are disjoint, it is enough to require that, x and y being two edges and m_x,m_y their associated variables, the inequality m_x + m_y \leq 1 is satisfied, as we are sure that the two of them can not both belong to the matching. Hence, we are able to write the MILP we want. However, the number of inequalities can be easily decreaded by noticing that two edges can not be taken simultaneously inside a matching if and only if they have a common endpoint v. We can then require instead that at most one edge incident to v be taken inside the matching (which is a linear constraint). We will be solving : .. MATH:: \text{Max : }&\sum_{e\in E(G)}m_e\\ \text{Tel que : }&\forall v, \sum_{e\in E(G)\atop v\sim e} m_e \leq 1\\ Let us write the Sage code of this MILP:: sage: g = graphs.PetersenGraph() sage: p = MixedIntegerLinearProgram() sage: matching = p.new_variable(binary = True) .. link :: sage: p.set_objective(sum( matching[e] ...                   for e in g.edges(labels = False))) .. link :: sage: for v in g: ...      p.add_constraint(sum( matching[e] ...           for e in g.edges_incident(v, labels = False)) <= 1) .. link :: sage: p.solve() 5.0 .. link :: sage: matching = p.get_values(matching) sage: [e for e,b in matching.iteritems() if b == 1] [(0, 1), (6, 9), (2, 7), (3, 4), (5, 8)] Flows ----- Yet another fundamental algorithm in graph theory : maximum flow ! It consists, given a directed graph and two vertices s, t, in sending a maximum *flow* from s to t using the edges of G, each f them having a maximal capacity. .. image:: lp_flot1.png :align: center The definition of this problem is almost its LP formulation. We are looking for real values associted to each edge, which would representthe intensity of flow going through them, under two types of constraints: * The amount of flow arriving on a vertex (different from s or t) is equal to the amount of flow leaving it * The amount of flow going through an edge is bounded by the capacity of this edge This being said, we but have to maximize the amount of flow leaving s : all of it will end up in t, as the other vertices are sending just as much as they receive. We can model the flow problem with the following LP .. MATH:: \text{Max : }&\sum_{sv\in G}f_{sv}\\ \text{Tel que : }&\forall v\in G, {v\neq s\atop v\neq t}, \sum_{vu\in G}f_{vu} - \sum_{uv \in G}f_{uv} = 0\\ &\forall uv\in G, f_{uv} \leq 1\\ We will he solve the flow problem on an orienration of Chvatal's Graph, in which all the edges have a capacity of 1:: sage: g = graphs.ChvatalGraph() sage: g = g.minimum_outdegree_orientation() .. link :: sage: p = MixedIntegerLinearProgram() sage: f = p.new_variable() sage: s, t = 0, 2 .. link :: sage: for v in g: ...      if v != s and v != t: ...          p.add_constraint( ...              sum( f[(v,u)] for u in g.neighbors_out(v)) ...             -sum( f[(u,v)] for u in g.neighbors_in(v)) == 0) .. link :: sage: for e in g.edges(labels = False): ...      p.add_constraint( f[e] <= 1 ) .. link :: sage: p.set_objective(sum( f[(s,u)] for u in g.neighbors_out(s))) .. link :: sage: p.solve() 2.0 .. image:: lp_flot2.png :align: center Solvers ------- Sage solves linear programs by calling specific libraries. The following libraries are currently supported : * CBC _: A solver from from COIN-OR _ (CPL -- Free) CBC can be installed through the command install_package("cbc") * CPLEX _: A solver from ILOG _ (Proprietary, but free for researchers and students) * GLPK _: A solver from GNU `_ (GPL3, Free) This solver is installed by default with Sage Installing CPLEX ---------------- ILOG CPLEX being Proprietary -- you must be in possession of several files to use it through Sage: * A valid license file * A compiled version of the CPLEX library (usually named libcplex.a) * The header file cplex.h The license file path must be set the value of the environment variable ILOG_LICENSE_FILE. For example, you can write :: export ILOG_LICENSE_FILE=/path/to/the/license/ilog/ilm/access_1.ilm at the end of your .bashrc file. As Sage also needs the files libcplex.a and cplex.h, the easiest way is to create symbolic links toward these files in the appropriate directories : * libcplex.a -- in SAGE_ROOT/local/lib/, type :: ln -s /path/to/lib/libcplex.a . * cplex.h -- in SAGE_ROOT/local/include/, type :: ln -s /path/to/include/cplex.h . Once this is done, and as CPLEX is used in Sage through the Osi library, which is part of the Cbc package, you can type:: sage: install_package("cbc")  # not tested or, if you had already installed Cbc :: sage: install_package("cbc", force = True)  # not tested to reinstall it.