Solvers for constant sum games

[ptigwe]

Constant-sum games are known to be solvable in polynomial time by using a linear program. This patch includes a solver which constructs and solves the LPs using the LP solvers within Sage (see ​http://doc.sagemath.org/html/en/reference/numerical/sage/numerical/mip.html). It also makes use of the solver within gambit for such games.

Finally, an additional function was included which helps to convert games from the representation in sage to gambits representation (_gambit_)

[kcrisman]

Various comments:

• We have other LP solvers, might as well test that they actually work (with optional doctests of course). Surely Nathann will have interest in yet another use of them :)

• E.g. although of course you test the gambit LP solver it would be nice to test the syntax here

  +        INPUT:
  +
  +        - ``solver`` - the solver to be used to solve the LP.
  +
  +          * ``'gambit'`` - This uses the solver included within the gambit library to
  +            create and solve the LP.
  

  same with other solvers.

• In _solve_gambit_LP do you need

  +        if not self.is_constant_sum():
  +            raise ValueError("Input game needs to be a two player constant sum game")
  

  since presumably this is already tested for end users in _solve_LP, or do you think this sort of double-checking is needed? (In which case you might want to test both of those branches.)

• What sort of error is raised if gambit isn't available for these LP things? Does it tell you to use gambit or does it say None has no such attribute or something?

• The option as_integer is neither documented nor doctested.

  +    def _as_gambit_game(self, as_integer=False):
  +        r"""
  +        Creates a Gambit game from a ``NormalFormGame`` object
  

• return c.numpy().max() == c.numpy().min() - is there no way to do this without using/importing numpy? It would be nice to not have to use it - or is it slower to use Sage proper?

• I don't mind in principle using the gambit conversion, obviously that is better when factored out, but then what happens to maximization in that case? Like

              sage: c._solve_LCP(maximization=True) # optional - gambit
              [[(0.0, 1.0), (0.0, 1.0)]]
  

  presumably still passes but what if one changed that to False?

• Is this going to be more efficient in the constant-sum case even if lrs is installed? I just don't know the answer to relative efficiency here; presumably LP isn't always faster, even if often, but I don't know anything about lrs (point-counting?) either.

  +            if self.is_constant_sum():
  +                algorithm = "lp-glpk"
  

• At this point there are so many options maybe one should also check for an invalid (read: mistyped) algorithm.

          if algorithm.startswith('lp-'):
              return self._solve_LP(solver=algorithm[3:])
  
          if algorithm == "enumeration":
              return self._solve_enumeration(maximization)
  

  and then else: blow up with a useful message

Finally, I haven't yet actually pulled anything nor verified that the MILP in question will solve constant-sum games, so that would remain as well to review this. Should be fun to get in, though, great to see stuff coming to Trac so early from the GSOC!!!

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​29d4a7d	Added tests for PPL and Coin-OR solvers
​6138791	Raise error if gambit isn't installed
​0029b3f	Improved documentation and doctests for _as_gambit_game
​82da1bf	Raise error upon wrong solver being passed
​d957179	Update check for constant sum 'is_constant_sum'

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​bb4eca8

Updated doctests for '_solve_LP'

[ptigwe]

Thanks for your comments. I've implemented most of your comments, and noted a few extra things below which would be done upon feedback. If I've missed anything please let me know.

Replying to kcrisman:

Various comments:

• We have other LP solvers, might as well test that they actually work (with optional doctests of course). Surely Nathann will have interest in yet another use of them :)

Done some doctests for PPL and Coin-OR solvers. Tests would be included for CVXOPT once #18572 is done.

​29d4a7d

Added tests for PPL and Coin-OR solvers

• In _solve_gambit_LP do you need

  +        if not self.is_constant_sum():
  +            raise ValueError("Input game needs to be a two player constant sum game")
  

  since presumably this is already tested for end users in _solve_LP, or do you think this sort of double-checking is needed? (In which case you might want to test both of those branches.)

I included it again just in case if the _solve_gambit_LP function was called externally without going through _solve_LP. If there is no need to retest just for this reason, then I'm happy to take it out.

• What sort of error is raised if gambit isn't available for these LP things? Does it tell you to use gambit or does it say None has no such attribute or something?

Just added a ValueError to be raised which is quite similar to what you get by trying to solve the game with algorithm='LCP' option.

​6138791

Raise error if gambit isn't installed

• return c.numpy().max() == c.numpy().min() - is there no way to do this without using/importing numpy? It would be nice to not have to use it - or is it slower to use Sage proper?

Currently, this compares all entries of the matrix and makes sure it is within sys.float_info.epsilon of the first element.

​d957179

Update check for constant sum 'is_constant_sum'

• I don't mind in principle using the gambit conversion, obviously that is better when factored out, but then what happens to maximization in that case? Like

              sage: c._solve_LCP(maximization=True) # optional - gambit
              [[(0.0, 1.0), (0.0, 1.0)]]
  

  presumably still passes but what if one changed that to False?

Currently, we are considering moving the maximization option into the constructor of the class.

• Is this going to be more efficient in the constant-sum case even if lrs is installed? I just don't know the answer to relative efficiency here; presumably LP isn't always faster, even if often, but I don't know anything about lrs (point-counting?) either.

  +            if self.is_constant_sum():
  +                algorithm = "lp-glpk"
  

The LP solvers would probably be faster primarily because lrs enumerates all possible extreme Nash equilibria in a game, whereas the LP method simply finds one Nash equilibrium in the constant-sum game.

• At this point there are so many options maybe one should also check for an invalid (read: mistyped) algorithm.

          if algorithm.startswith('lp-'):
              return self._solve_LP(solver=algorithm[3:])
  
          if algorithm == "enumeration":
              return self._solve_enumeration(maximization)
  

  and then else: blow up with a useful message

​6138791

Raise error if gambit isn't installed

There are two possible ValueError's that could be raised. The first comes from NormalFormGame upon entering a solver of the wrong format, and the second is from MixedIntegerLinearProgram if the LP solver is invalid.

[kcrisman]

Thanks for your comments. I've implemented most of your comments, and noted a few extra things below which would be done upon feedback. If I've missed anything please let me know.

Great, very quick work.

I included it again just in case if the _solve_gambit_LP function was called externally without going through _solve_LP. If there is no need to retest just for this reason, then I'm happy to take it out.

Well, usually such underscore methods aren't "publicly" available. Vince, what do you think?

• return c.numpy().max() == c.numpy().min() - is there no way to do this without using/importing numpy? It would be nice to not have to use it - or is it slower to use Sage proper?

Currently, this compares all entries of the matrix and makes sure it is within sys.float_info.epsilon of the first element.

​d957179

Update check for constant sum 'is_constant_sum'

But which one is in principle faster or better? I don't mind using numpy as long as importing it doesn't cause problems in speed, if it's better (which perhaps it is).

Currently, we are considering moving the maximization option into the constructor of the class.

Fine, but this is currently breaking functionality. So either you have to do that here, or make that change a prereq to this ticket, or something else. My recommendation is to just leave it here for now and then deal with the class constructor bit in a separate ticket (to make things as orthogonal as possible).

The LP solvers would probably be faster primarily because lrs enumerates all possible extreme Nash equilibria in a game, whereas the LP method simply finds one Nash equilibrium in the constant-sum game.

Ah.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​60efdc7

Fixed '_as_gambit_game' to support 'maximization' parameter

[ptigwe]

I included it again just in case if the _solve_gambit_LP function was called externally without going through _solve_LP. If there is no need to retest just for this reason, then I'm happy to take it out.

Well, usually such underscore methods aren't "publicly" available. Vince, what do you think?

Actually, gambit performs it's own check as well, which makes three checks in total. So I've removed the one within _solve_gambit_LP, and added a doctest for the gambit error.

• return c.numpy().max() == c.numpy().min() - is there no way to do this without using/importing numpy? It would be nice to not have to use it - or is it slower to use Sage proper?

Currently, this compares all entries of the matrix and makes sure it is within sys.float_info.epsilon of the first element.

​d957179

Update check for constant sum 'is_constant_sum'

But which one is in principle faster or better? I don't mind using numpy as long as importing it doesn't cause problems in speed, if it's better (which perhaps it is).

The current implementation is faster than the numpy implementation. I ran some bench marks using timeit and got 177 µs per loop for the current implementation vs 992 ms per loop for the numpy implementation using a matrix of size 1000x1000.

Currently, we are considering moving the maximization option into the constructor of the class.

Fine, but this is currently breaking functionality. So either you have to do that here, or make that change a prereq to this ticket, or something else. My recommendation is to just leave it here for now and then deal with the class constructor bit in a separate ticket (to make things as orthogonal as possible).

Integrated the maximization parameter into _as_gambit_game.

[kcrisman]

Great. Here is what I think still needs to happen.

• You should add an example testing the maximization=False
• This error looks messed up because of the extra spaces - check for others like this:

  ValueError: The Gambit implementation of LCP only
                                       allows for integer valued payoffs.
                                       Please scale your payoff matrices.
  

• You should fix the following formatting

  +        INPUT:
  +        - ``as_integer`` - Boolean value which states whether the gambit representation should have
  +                           the payoffs represented as integers or decimals.
  

  (there should be a blank line between INPUT: and the rest, I think) even though it is an underscore method and won't appear in documentation.
• You should decide whether you want # optional - Coin or # optional - cbc
• Is PPL or CVXOPT really an optional thing? I think they are standard Sage packages...
• It would be really nice to have at least one example in the doc of a solved (not just constructed, as in the documentation of is_constant_sum) constant-sum non-zero-sum game.
• I (or another reviewer) need to check that the MILP is the right one
• I (or another reviewer) need to check that documentation builds right and looks good
• I (or another reviewer) need to check that the tests work right - guess I had better start installing some packages :)
• I (or another reviewer) need to do some spot checks of everything

Which is all not too hard, we're almost there.

What is the next project on the GSOC timeline?

[kcrisman]

I cannot figure out how to get the changes in the doc built, it's maddening. I'll have to assume it's okay and I didn't miss anything.

The following could be improved, it really looks pretty meager. Not necessarily for this ticket, as definitely pre-existing - Vince, I'll blame you for this :)

class NormalFormGame(SageObject, MutableMapping):
    r"""
    An object representing a Normal Form Game. Primarily used to compute the
    Nash Equilibria.

    INPUT:

    - ``generator`` - Can be a list of 2 matrices, a single matrix or left
                      blank.

    """

It should at least tell how to get to more documentation from there.

The following

+        p = MixedIntegerLinearProgram(maximization=False, solver=solver)
+        y = p.new_variable(nonnegative=True)
+        v = p.new_variable(nonnegative=False)
+        p.add_constraint(self.payoff_matrices()[0] * y - v[0] <= 0)
+        p.add_constraint(matrix([[1] * strategy_sizes[0]]) * y == 1)
+        p.set_objective(v[0])

looks good though I would prefer to use maximization and swap that first constraint or whatever is appropriate, but presumably this is the industry standard to minimize.

So... is the reason for doing this only for constant-sum games because it's only known to be in P for them? Is it potentially still faster even for non-constant-sum games?

Doctests pass.

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​0e300ae	Fixed indentation and removed incorrect error
​313f42c	Updated tests for cbc and PPL
​a24c7dd	Included tests for constant-sum non-zero sum game and included maximization in the LP solver

[ptigwe]

• You should add an example testing the maximization=False

There's currently one at the start of obtain_nash, showing that it's possible for the equilibrium found could be different. Do you want me to do something similar for all the methods, or would the one be enough?

• This error looks messed up because of the extra spaces - check for others like this:

  ValueError: The Gambit implementation of LCP only
                                       allows for integer valued payoffs.
                                       Please scale your payoff matrices.
  

Actually, I was supposed to remove this error earlier on as it doesn't hold. I removed the documentation but forgot to remove the actual error.

• You should decide whether you want # optional - Coin or # optional - cbc
• Is PPL or CVXOPT really an optional thing? I think they are standard Sage packages...

I've removed #optional - PPL.

• I (or another reviewer) need to check that the MILP is the right one
• I (or another reviewer) need to check that documentation builds right and looks good
• I (or another reviewer) need to check that the tests work right - guess I had better start installing some packages :)
• I (or another reviewer) need to do some spot checks of everything

Which is all not too hard, we're almost there.

What is the next project on the GSOC timeline?

Coming up next would be the Lemke-Howson algorithm for solving 2-player games.

[ptigwe]

The following could be improved, it really looks pretty meager. Not necessarily for this ticket, as definitely pre-existing - Vince, I'll blame you for this :)

class NormalFormGame(SageObject, MutableMapping):
    r"""
    An object representing a Normal Form Game. Primarily used to compute the
    Nash Equilibria.

    INPUT:

    - ``generator`` - Can be a list of 2 matrices, a single matrix or left
                      blank.

    """

It should at least tell how to get to more documentation from there.

OK. I was planning on opening a ticket, towards the end of GSOC, which would address the documentation as a whole. In the mean time, changes to the documentation would be based on changes made to the code.

The following

+        p = MixedIntegerLinearProgram(maximization=False, solver=solver)
+        y = p.new_variable(nonnegative=True)
+        v = p.new_variable(nonnegative=False)
+        p.add_constraint(self.payoff_matrices()[0] * y - v[0] <= 0)
+        p.add_constraint(matrix([[1] * strategy_sizes[0]]) * y == 1)
+        p.set_objective(v[0])

looks good though I would prefer to use maximization and swap that first constraint or whatever is appropriate, but presumably this is the industry standard to minimize.

So... is the reason for doing this only for constant-sum games because it's only known to be in P for them?

Yeah.

Is it potentially still faster even for non-constant-sum games?

Not sure if you are asking about using the LP for non-constant-sum games or the general classification of finding a Nash in a general game. In case if it was a bit of both:

• LP's aren't guaranteed to find a Nash equilibrium in a two player non-constant-sum game.
• Finding a Nash is PPAD-complete. Even the problem of approximation is hard.

Doctests pass.

That's Good.

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

OK.

[kcrisman]

OK. I was planning on opening a ticket, towards the end of GSOC, which would address the documentation as a whole.

Sounds fair.

Not sure if you are asking about using the LP for non-constant-sum games or the general classification of finding a Nash in a general game. In case if it was a bit of both:

• LP's aren't guaranteed to find a Nash equilibrium in a two player non-constant-sum game.

Very interesting!

[vinceknight]

Hi both,

Apologies for my silence (on organisation committee of a conference that has been running this week so have been pretty busy). Apologies also if as a result my comments below don't make full sense (in case I missed something).

Replying to kcrisman:

I cannot figure out how to get the changes in the doc built, it's maddening. I'll have to assume it's okay and I didn't miss anything.

The following could be improved, it really looks pretty meager. Not necessarily for this ticket, as definitely pre-existing - Vince, I'll blame you for this :)

class NormalFormGame(SageObject, MutableMapping):
    r"""
    An object representing a Normal Form Game. Primarily used to compute the
    Nash Equilibria.

    INPUT:

    - ``generator`` - Can be a list of 2 matrices, a single matrix or left
                      blank.

    """

It should at least tell how to get to more documentation from there.

This is certainly due to me, it happened as a result of moving the documentation that was there to the front matter, I think I perhaps did not fully understand :) Perhaps a ticket could be opened about the documentation (as Tobenna suggested towards the end of gsoc) that fully describes what we want the docs to look like :)

Replying to ptigwe:

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

OK.

I think it could be worth doctesting also? (So showing that all equilibria are find by some algorithms and not by LP etc...)

Great work on this Tobenna, really looking solid and thanks to Karl for reviewing (as always this is greatly appreciated).

[kcrisman]

This is certainly due to me, it happened as a result of moving the documentation that was there to the front matter, I think I perhaps did not fully understand :) Perhaps a ticket could be opened about the documentation (as Tobenna suggested towards the end of gsoc) that fully describes what we want the docs to look like :)

Actually, I'll open it now and set it to sage-pending so we don't forget. #18609.

I think it's worth pointing out somewhere in the documentation that the LP approach will give one NE but not all of them, should there be more than one. (Constant-sum games can have more than one NE, right? I guess the trivial game is an example.)

OK.

I think it could be worth doctesting also? (So showing that all equilibria are find by some algorithms and not by LP etc...)

Okay, so we're agreed on this.

Another small point - no maximization here.

        if algorithm.startswith('lp-'):
            return self._solve_LP(solver=algorithm[3:])

Of course, at some point this is pedantic if you are planning on removing it - but actually, given that it pre-existed this ticket, I guess you'll have to deprecate that keyword.

You should add an example testing the maximization=False

There's currently one at the start of obtain_nash, showing that it's possible for the equilibrium found could be different. Do you want me to do something similar for all the methods, or would the one be enough?

What I meant was one for the new LP solver functionality.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2f44485	Tests for single / multiple Nash equilibria
​fb4461c	Fixed minor error with LP solver

[ptigwe]

Replying to kcrisman:

Another small point - no maximization here.

        if algorithm.startswith('lp-'):
            return self._solve_LP(solver=algorithm[3:])

Of course, at some point this is pedantic if you are planning on removing it - but actually, given that it pre-existed this ticket, I guess you'll have to deprecate that keyword.

You should add an example testing the maximization=False

There's currently one at the start of obtain_nash, showing that it's possible for the equilibrium found could be different. Do you want me to do something similar for all the methods, or would the one be enough?

What I meant was one for the new LP solver functionality.

Dima, Vince and I have thought and discussed about this issue for a while, and have decided to remove maximization completely. So the next few patches should be addressing this.

[kcrisman]

Okay. Probably best is to have that be a dependency of this ticket... And if you end up not deprecating this and just remove it (which could conceivably be reasonable in the particular context) be sure to have plenty of good arguments backing up that decision "for posterity's sake".

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e4107dc

Updated tests for normal form games

[ptigwe]

Besides the deprecation (which would be done in a separate ticket), I believe that's everything for this one. If I've missed anything please let me know.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c225b92

Remove maximization from LP functions as it is going to be deprecated

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​93229b7

Revert "Remove maximization from LP functions as it is going to be deprecated"

[tscrim]

Instead of calling the method _as_gambit_game, to be consistent with other parts of Sage, I would call it _gambit_ (like how the _gap_ method returns a representation of the object in GAP). Also the doc for INPUT: for that method is misaligned.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​ddd6f7e

Renamed '_as_gambit_game' to '_gambit_' and fixed 'INPUT' indentation issues

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​6e2aae5	Merge branch 'develop' into gt_extension
​92345cc	Modified the '_gambit_' function to support n-player games
​2c6aee7	Update doctests
​06d6b4c	Updated doctests of `catalog` to use `enumeration`

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​2c4d951

Updated front matter

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​e538b14

Added name to AUTHORS

[kdilks]

More doctests need to be marked as optional.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​51912a5	Updated optional flags
​90f7051	Merge branch 'develop' into gt_extension

[mderickx]

Needs to be rebased on sage8.1.beta3

[tscrim]

Rebased version.

---

New commits:

​9b4a9cb

Rebased on 8.1.beta9.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​9e009fb

Some small documentation updates.

[tscrim]

I believe this is essentially reviewed, so perhaps the only thing left to check are my little round of tweaks.

[dimpase]

This still needs the following for doc building and to pass the tests

--- a/src/sage/game_theory/normal_form_game.py
+++ b/src/sage/game_theory/normal_form_game.py
@@ -216,7 +216,7 @@ currently available:
 
 * ``'lp-*'``: A solver for constant sum 2 player games using linear
   programming. This contructs a
- `:mod:MixedIntegerLinearProgram <sage.numerical.MILP>` using the
+  `:mod:MixedIntegerLinearProgram <sage.numerical.MILP>` using the
   solver which was passed in as part of the algorithm string to solve
   the linear programming representation of the game, for instance,
   ``'lp-glpk'`` would make use of the ``GLPK`` solver, while
@@ -568,6 +568,7 @@ Here is an example with the trivial game where all payoffs are 0::
     (
     [0 0 0]  [0 0 0]
     [0 0 0]  [0 0 0]
+    [0 0 0], [0 0 0]
     )
     sage: g.obtain_nash(algorithm='enumeration')
     [[(0, 0, 1), (0, 0, 1)], [(0, 0, 1), (0, 1, 0)], [(0, 0, 1), (1, 0, 0)],

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​c2a1187

fixed doc building and a test

[tscrim]

Thank you.

[dcoudert]

The last commit has introduced a small mistake. One should do:

- `:mod:MixedIntegerLinearProgram <sage.numerical.MILP>` using the
+ :mod:`MixedIntegerLinearProgram <sage.numerical.MILP>` using the

I think that using algorithm = "lp-glpk" or algorithm = "lp-gambit" is not the best choice. It forces to specify the LP solver, while we usually let Sage use the default LP solver (GLPK, Cplex, ...). Why not using 2 parameters: algorithm="lp" and solver=None (or "gambit" or "gplk"or ...) ?

Typically, we usually do:

- def _solve_LP(self, solver='glpk', maximization=True):
+ def _solve_LP(self, solver=None, maximization=True):

[git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​f5b73ea

fixed doc building and a test

[dimpase]

Replying to git:

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

​f5b73ea

fixed doc building and a test

this fixes the wrongly placed backtick for :mod:...

[dimpase]

Replying to dcoudert:

I think that using algorithm = "lp-glpk" or algorithm = "lp-gambit" is not the best choice. It forces to specify the LP solver, while we usually let Sage use the default LP solver (GLPK, Cplex, ...). Why not using 2 parameters: algorithm="lp" and solver=None (or "gambit" or "gplk"or ...) ?

This is more subtle, as gambit is not one of LP-solvers supported by the Sage's LP backend, and thus the change you propose would introduce a bit of dissonance, too. I would prefer to have such an improvement worked on on a followup ticket.

[tscrim]

Since this is being introduced on this ticket, I think we should try and decide now rather than later, especially since that would likely mean deprecations.

gambit does apparently come with a solver, so I feel it is only very mildly dissonant. I am working on David's suggestion.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​fab0962

Using a solver argument for LP algorithm.

[tscrim]

Done. Now that it is done, I think this is a better approach.

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​80203f8

typo fixed

[git]

Branch pushed to git repo; I updated commit sha1. New commits:

​156ea0f

some improvements to docs and tests

[dcoudert]

I think it's better now. I let you conclude on the status of this ticket since I have not evaluated the other parts.