When we ask friends if we can stay at their place, we prefer them to say yes without asking details such as for how long. Yet, if the answer is going to be no, then we often prefer them to seek more information from us first. At first glance, this situation seems very different from how we react when we are in an exclusive relationship and our partner flirts with someone else. However, a probability-based analysis with Bayesian game theory shows that each involves differing degrees of manipulation and preferential interaction, MIT researchers report in a recent paper.
MIT postdoc Alfonso Pérez-Escudero and colleagues analyzed how these manipulation and preferential interaction mechanisms play out in “the envelope game,” a framework developed by Harvard University researchers Martin Nowak, Erez Yoeli, and Moshe Hoffman. “These are two situations that, in principle, I wouldn’t put together, and thanks to the framework that these researchers developed, we realized that they belong to the same family of situations,” Pérez-Escudero says. The original framework contained the manipulative mechanism but not the preferential interaction mechanism. “Our contribution is to realize that this family has two sub-families that can be mixed. We developed a generalization, creating a model that can describe both of these mechanisms at the same time and that contains the original model as a particular case,” he says.
The paper, co-authored by Pérez-Escudero, postdoc Jonathan Friedman, and MIT Latham Family Career Development Associate Professor of Physics Jeff Gore, was published in the Proceedings of the National Academy of Sciences on Dec. 6, 2016. The Gore Laboratory, in the Physics of Living Systems group at MIT, more often uses game theory to explain evolutionary dynamics such as cooperation among microbes. “Here we use the same math that you can use to describe evolution in biology to describe human behavior and human psychology, building a unifying framework between biological problems and human ones,” Gore says.
An unequal game
In the envelope game, each player has a potential payoff, but the first player’s actions determine the second player’s payoff, so their roles are unequal, or asymmetric. The envelope contains a bonus prize, which is sometimes of low value and sometimes of high value, and Player 1 can choose whether to look in the envelope. After either looking in the envelope or choosing not to look, Player 1 will decide next whether to “cooperate,” in which case both players get a payoff, or to “defect,” in which case only Player 1 gets a payoff and Player 2 takes a loss. Player 2’s only choice is whether to continue the game or to quit.
“The player who can cooperate or defect has power over payoffs in the current round, while the other player has the power to continue the game for more rounds or stop,” Pérez-Escudero explains. “It’s unsurprising that Player 2 ends the game in retaliation if Player 1 defects. The surprising part is that Player 2 may also end the game if Player 1 cooperates — just because he looked in the envelope — even though looking has no effect on Player 2’s payoffs.”
Strategic choice, open signal
Each player is assigned a strategy profile, which is a set of rules that tells each player what to do in each situation. Since both players stand to win more by continuing the game, Player 2’s choice whether to continue or to end the game will influence player 1’s strategy. Players who always cooperate are said to be “reliable,” whereas players who sometimes, or always, defect are “unreliable.” When a set of strategies for Player 1 and Player 2 reaches a balanced state where neither player will benefit by following a different strategy, it is said to be in “equilibrium,” in mathematical terms, either a Nash equilibrium or a sequential equilibrium.
The player who looks in the envelope is sending a signal to the other player. “By opening the envelope, she is telling us something,” Pérez-Escudero says. “This means that her decision wasn’t very clear. She wasn’t 100 percent sure she was going to cooperate, nor 100 percent sure she was going to defect. She needed this piece of information to make her decision. So the price of learning the contents of the envelope is giving away some information about herself.” The intuition, then, is that the other player will prefer not to interact with someone who was not sure from the beginning, because she might change her mind if the conditions change slightly. “But this is what piqued our curiosity,” he says. “In the original paper, both players know each other perfectly — the only uncertainty is about the content of the envelope. And if I already know that you are an unreliable cooperator, I will not learn anything when I see that you look in the envelope; looking should play no role. But the results were there: The envelope game does have a Nash equilibrium in which looking plays a fundamental role.” Something was missing.
By analyzing the game more closely, the MIT researchers realized that a different mechanism was driving the results: By threatening to end the game if Player 1 looks, Player 2 can force her to make a blind decision. This manipulative mechanism makes looking a key element of the game’s Nash equilibrium, even if both players know each other perfectly. “But this is not what happens in real life. Uncertainty is everywhere, and even if we know a person, we are never sure of their true feelings and thoughts. So we implemented this uncertainty into the model, turning it into a Bayesian game,” Pérez-Escudero says. [In everyday life, applying Bayesian rules is how email programs filter out spam.]
Computer simulations
Using computer simulations, Pérez-Escudero modeled how different strategies play out over several thousand rounds, which yields data for about 100,000 to 1 million possible combinations. The model runs a mathematical formula to simulate the repeated games and differing strategies. What he found is that when Player 1 always acts in the same way, only manipulation can make looking matter in the game. Player 2 accomplishes this manipulation by ending the game if Player 1 looks, which effectively punishes Player 1 by denying her any future gains and also protects Player 2 against any further losses. But where the game introduces two varieties of Player 1 with different payoffs and strategies, Player 2 will see “favorable” and “unfavorable” types, and pay attention to looking as a cue to tell them apart. In this scenario, no punishment is required.
The mathematical formula, which is called a replicator equation, comes from evolutionary biology. “Imagine you have a population with 1,000 people that are playing slightly different strategies; those with more successful strategies are going to have more children, and they are going in the end to take over the population. The replicator equation was designed to describe this kind of situation, but it was found later that it can also describe cultural evolution, where a given idea (or behavior) can be learned and copied, making it a powerful tool to analyze human behavior. But to use it properly, one needs to enumerate all the possible strategies that can exist in the game. If I enumerate all these strategies, then the replicator equation can tell me who wins,” Pérez-Escudero explains.
But simulations alone would not be enough. The envelope game has infinite possible strategies, so it’s just not possible to enumerate them all. Simulations were therefore complemented by a different tool from game theory called the one-shot deviation principle, which acts to put a limit on what otherwise would be infinite calculations in order to draw meaningful conclusions. “Thanks to this principle, we can prove that a strategy is optimal even if we don’t know what other strategies are out there. You start from your strategy and test every decision you make, one by one. If you cannot benefit by deviating from the strategy in any single decision, then it is a best response and potentially part of a Nash equilibrium, or in our game, a refinement called sequential equilibrium,” he says. “Simulations, even if they cannot prove the equilibrium, were still useful to check that the equilibriums we were finding were also stable.”
Manipulation versus preference
These mathematical models neatly simulate personal interactions, where both manipulation and preferential interactions play a role — often together. “Take for example an exclusive couple relationship. If I have an exclusive relationship with my girlfriend and I flirt with other people, I can expect my girlfriend to punish me. She can get very angry; she can leave me. In this case, there is true leverage from one person to the other, and then it’s very likely that the manipulative mechanism is playing a role.” The preferential interaction mechanism can also play a role here because one partner’s decision to flirt also informs the other that her partner is perhaps not very invested in the relationship. “Maybe she would prefer another person who is more invested in the relationship,” he says. “Here there are these two mechanisms. On the one hand, she is learning something about me and maybe she prefers not to interact more with me because of what she learns. On the other hand, she has the power to punish me if I do something she doesn’t want me to do,” Pérez-Escudero says.
Another key finding of the study is that the preferential interactions mechanism can give rise to the opposite effect: preference for looking. The defining characteristic is whether Player 1 ends up cooperating or defecting. “If Player 1 cooperates, I prefer her to cooperate without looking, because she’s a reliable cooperator. If Player 1 defects, I prefer her to look, because then she could be an unreliable defector, and I can still hope she will cooperate in the future,” Pérez-Escudero explains. “I think this connects with real-life situations. If you ask ‘Can you do me a favor?’ it would be very rude that I just say no. Instead, even if I’m confident I will not grant the favor, I will first ask what favor is it, and then present an excuse. My asking here would be a false signal that prevents you from realizing that I’m such a bad person that I would not grant you even the smallest favor.”
Commenting on the new MIT findings, Moshe Hoffman, a research scientist and lecturer at Harvard's Program for Evolutionary Dynamics, says, “The model helps us understand why we trust more those who don't look at the costs and benefits before deciding whether to cooperate, and more generally why we value principled behavior above strategic calculated behavior.”
“This model is a solid contribution to our understanding of principles of behavior, cooperation, and morality, and more generally fits within a wider literature that is important and insightful which uses game theoretic models and models of learning and evolutionary processes to understand puzzling aspects of human social behavior,” Hoffman says. “How else can we understand our social species if we don't try and uncover the hidden function behind what they do think and believe? And what better tools to do that than models of game theory, learning and evolutionary processes?”
This work was supported by an EMBO Postdoctoral Fellowship, Human Frontier Science Foundation Postdoctoral Fellowship, and the Paul Allen Family Foundation.
