Program equilibrium is a game-theoretic solution concept for a scenario in which players submit computer programs to play the game on their behalf and the programs can read each other's source code. The term was introduced by Moshe Tennenholtz in 2004. The same setting had previously been studied by R. Preston McAfee, J. V. Howard and Ariel Rubinstein.
Setting and definition The program equilibrium literature considers the following setting. Consider a normal-form game as a base game. For simplicity, consider a two-player game in which S 1 {\displaystyle S_{1}} and S 2 {\displaystyle S_{2}} are the sets of available strategies and u 1 {\displaystyle u_{1}} and u 2 {\displaystyle u_{2}} are the players' utility functions. Then we construct a new (normal-form) program game in which each player i {\displaystyle i} chooses a computer program p i {\displaystyle p_{i}} . The payoff (utility) for the players is then determined as follows. Each player's program p i {\displaystyle p_{i}} is run with the other program p − i {\displaystyle p_{-i}} as input and outputs a strategy s i {\displaystyle s_{i}} for Player i {\displaystyle i} . For convenience one also often imagines that programs can access their own source code. Finally, the utilities for the players are given by u i ( s 1 , s 2 ) {\displaystyle u_{i}(s_{1},s_{2})} for i = 1 , 2 {\displaystyle i=1,2} , i.e., by applying the utility functions for the base game to the chosen strategies. One has to further deal with the possibility that one of the programs p i {\displaystyle p_{i}} doesn't halt. One way to deal with this is to restrict both players' sets of available programs to prevent non-halting programs. A program equilibrium is a pair of programs ( p 1 , p 2 ) {\displaystyle (p_{1},p_{2})} that constitute a Nash equilibrium of the program game. In other words, ( p 1 , p 2 ) {\displaystyle (p_{1},p_{2})} is a program equilibrium if neither player i {\displaystyle i} can deviate to an alternative program p i ′ {\displaystyle p_{i}'} such that their utility is higher in ( p i ′ , p − i ) {\displaystyle (p_{i}',p_{-i})} than in ( p 1 , p 2 ) {\displaystyle (p_{1},p_{2})} . Instead of programs, some authors have the players submit other kinds of objects, such as logical formulas specifying what action to play depending on an encoding of the logical formula submitted by the opponent.
Different mechanisms for achieving cooperative program equilibrium in the Prisoner's Dilemma Various authors have proposed ways to achieve cooperative program equilibrium in the Prisoner's Dilemma.
Cooperation based on syntactic comparison Multiple authors have independently proposed the following program for the Prisoner's Dilemma:
algorithm CliqueBot(opponent_program): if opponent_program == this_program then return Cooperate else return Defect
If both players submit this program, then the if-clause will resolve to true in the execution of both programs. As a result, both programs will cooperate. Moreover, (CliqueBot,CliqueBot) is an equilibrium. If either player deviates to some other program p i {\displaystyle p_{i}} that is different from CliqueBot, then the opponent will defect. Therefore, deviating to p i {\displaystyle p_{i}} can at best result in the payoff of mutual defection, which is worse than the payoff of mutual cooperation. This approach has been criticized for being fragile. If the players fail to coordinate on the exact source code they submit (for example, if one player adds an extra space character), both programs will defect. The development of the techniques below is in part motivated by this fragility issue.
Proof-based cooperation Another approach is based on letting each player's program try to prove something about the opponent's program or about how the two programs relate. One example of such a program is the following:
algorithm FairBot(opponent_program): if there is a proof that opponent_program(this_program) = Cooperate then return Cooperate else return Defect
… excerpt ends here. Continue reading the full article.
