No description available.

Peer-to-peer (p2p) approaches are an increasingly effective way to deploy services. Popular examples include BitTorrent, Skype, and KaZaA. These approaches are attractive because they can be highly fault-tolerant, scalable, adaptive, and less expensive than a more centralized solution. Cooperation lies at the heart of these strengths. Yet, in settings where working together is crucial, a natural question is: "What if users stop cooperating?" After all, cooperative services are typically deployed over multiple administrative domains, and thus vulnerable to Byzantine failures and users who may act selfishly. This dissertation explores how to construct p2p systems to tolerate Byzantine participants while also incentivizing selfish participants to contribute resources. We describe how to balance obedience against choice in building a robust p2p live streaming system. Imposing obedience is desirable as it leaves little room for peers to attack or cheat the system. However, providing choice is also attractive as it allows us to engineer flexible and efficient solutions. We first focus on obedience by using Nash equilibria to drive the design of BAR Gossip, the first gossip protocol that is resilient to Byzantine and selfish nodes. BAR Gossip relies on verifiable pseudo-random partner selection to eliminate non-determinism, which can be used to game the system, while maintaining the robustness and rapid convergence of traditional gossip. A novel fair enough exchange primitive entices cooperation among selfish peers on short timescales, thereby avoiding the need for distributed reputation schemes. We next focus on tempering obedience with choice by using approximate equilibria to guide the construction of a novel p2p live streaming system. These equilibria allow us to design incentives to limit selfish behavior rigorously, yet provide sufficient flexibility to build practical systems. We show the advantages of using an [element of]-Nash equilibrium, instead of an exact Nash, to design and implement FlightPath, our live streaming system that uses bandwidth efficiently, absorbs flash crowds, adapts to sudden peer departures, handles churn, and tolerates malicious activity. / text

Στην εργασία αυτή ασχολούμαστε με το πρόβλημα της δρομολόγησης ενός συνόλου αιτήσεων επικοινωνίας σε WDM (Wavelength Division Multiplexing) πλήρως οπτικά δίκτυα από την άποψη της θεωρίας παιγνίων. Αν θεωρήσουμε κάθε αίτηση δρομολόγησης (ζεύγος κόμβων αφετηρία-προορισμός) ως παίκτη, τότε μία στρατηγική περιλαμβάνει ένα μονοπάτι από τον κόμβο-αφετηρία στον κόμβο-προορισμό και μία συχνότητα (χρώμα). Λαμβάνοντας υπόψη τον περιορισμό ότι δύο παίκτες δεν μπορούν να χρησιμοποιήσουν την ίδια συχνότητα στην ίδια ακμή, θεωρούμε ότι το κόστος δύο αλληλοσυγκρουόμενων στρατηγικών είναι απαγορευτικά μεγάλο. Στο παραπάνω πλαίσιο, μελετάμε διάφορες φυσικές συναρτήσεις κόστους επικεντρώνοντας στην ύπαρξη αμιγών σημείων ισορροπίας Nash και στην υπολογιστική πολυπλοκότητα αναγνώρισης και υπολογισμού τους. / We consider the problem of routing a number of communication requests in WDM (wavelength division multiplexing) all-optical networks from the standpoint of game theory. If we view each routing request (pair of source-target nodes) as a player, then a strategy consists of a path from the source to the target and a frequency (color). To reflect the restriction that two requests must not use the same frequency on the same edge, conflicting strategies are assigned a prohibitively high cost. Under this formulation, we consider several natural cost functions focusing on the existence of Nash equilibria and on the complexity of recognizing and computing them.

Cette thèse se focalise sur les jeux de potentiel et une généralisation d'un jeu d'ordonnancement dans un graphe que nous avons appelé jeu de placement.Dans ce jeu, le coût d'un joueur est impacté par son voisinage.Nous pouvons illustrer cela avec un exemple : le placement de joueurs dans un train, pour lesquels la présence de voisins directs influe sur le bien-être.Les résultats de cette thèse se divisent en deux parties.Tout d'abord, nous étudions ces jeux en considérant l'existence et les propriétés de structure des équilibres.Nous nous posons la question fondamentale de savoir s'il existe des équilibres de Nash dans le jeu de placement.Si tel est le cas, nous tachons de déterminer si ces équilibres sont facilement calculables.Dans le cas où il n'existe pas d'équilibre nous prouvons la NP-complétude du problème.Dans un second temps nous nous intéressons à la notion de calcul distribué d'équilibre de Nash dans des jeux de placement.En particulier nous considérons un jeu basé sur le problème de Max-Cut, qui a été plus étudié en théorie des graphes.Cela nous a permis d'étendre nos travaux à une application aux réseaux mobiles pour la gestion d'interférences dans les réseaux sans fils.Nous avons pu, pour les différents jeux, mettre en place des algorithmes distribués de calcul d'équilibres et étudier leur convergence.Parallèlement, nous avons étendu les travaux de Max-Cut à un problème de sélection d'offre de qualité de service parmi divers fournisseurs d'accès.Nous comparons les performances d'algorithmes de calcul distribué d'équilibres et de minimisation de regret. / This thesis focuses on potential games and a generalized load balancing game in a graph we called placement game.In this game, the cost of a player is affected by its neighbors.We can illustrate this with an example: the placement of players on a train, where the presence of direct neighbors affects their well-being.The results of this thesis are divided into two parts.First, we study these games considering the existence and structural properties of equilibria.We ask ourselves the fundamental question of whether there are Nash equilibria in the placement game.If this is the case we aim to determine if they are easily calculable, if there is no such equilibria we prove the NP-completeness of the problem.Secondly we focus on the concept of distributed algorithms to compute Nash equilibria in placement games.In particular we consider a game based on the Max-Cut problem, which has been more frequently studied.This allowed us to expand our work to a mobile network application for managing interference in wireless networks.We were able, for those different games, to implement distributed algorithms to compute equilibria and study their convergence.Meanwhile, we have expanded the Max-Cut works with a selection of QoS offers problem from various network providers.We compare the performance of distributed algorithms and regret minimization.

Algorithmically designed reward functions can influence groups of learning agents toward measurable desired sequential joint behaviours. Influencing learning agents toward desirable behaviours is non-trivial due to the difficulties of assigning credit for global success to the deserving agents and of inducing coordination. Quantifying joint behaviours lets us identify global success by ranking some behaviours as more desirable than others. We propose a real-valued metric for turn-taking, demonstrating how to measure one sequential joint behaviour. We describe how to identify the presence of turn-taking in simulation results and we calculate the quantity of turn-taking that could be observed between independent random agents. We demonstrate our turn-taking metric by reinterpreting previous work on turn-taking in emergent communication and by analysing a recorded human conversation. Given a metric, we can explore the space of reward functions and identify those reward functions that result in global success in groups of learning agents. We describe 'medium access games' as a model for human and machine communication and we present simulation results for an extensive range of reward functions for pairs of Q-learning agents. We use the Nash equilibria of medium access games to develop predictors for determining which reward functions result in turn-taking. Having demonstrated the predictive power of Nash equilibria for turn-taking in medium access games, we focus on synthesis of reward functions for stochastic games that result in arbitrary desirable Nash equilibria. Our method constructs a reward function such that a particular joint behaviour is the unique Nash equilibrium of a stochastic game, provided that such a reward function exists. This method builds on techniques for designing rewards for Markov decision processes and for normal form games. We explain our reward design methods in detail and formally prove that they are correct.

Esta dissertação aborda a teoria dos jogos diferenciais em sua estreita relação com a teoria das equações de Hamilton-Jacobi (HJ). Inicialmente, uma revisão da noção de solução em teoria dos jogos é empreendida. Discutem-se nesta ocasião as idéias de equilíbrio de Nash e alguns de seus refinamentos. Em seguida, tem lugar uma introdução à teoria dos jogos diferenciais, onde noções de solução como a função de valor de Isaacs e de Friedman são discutidas. É nesta altura do trabalho que fica evidente a conexão entre este conceito de solução e a teoria das equações de Hamilton-Jacobi. Por ocasião desta conexão, é explorada a noção de solução clássica e é exposta uma demonstração do fato de que se um jogo diferencial possuir uma função de valor pelo menos continuamente diferenciável, esta será uma solução da equação de Hamilton-Jacobi associada ao jogo. Este resultado faz uso do princípio da programação dinâmica, devido a Bellman, e cuja demonstração está presente no texto. No entanto, quando a função de valor do jogo é apenas contínua, então embora esta não seja uma solução clássica da equação HJ associada a jogo, vemos que ela será uma solução viscosa, ou solução no sentido da viscosidade - e a esta altura são discutidos os elementos e propriedades desta classe de soluções, um teorema de existência e unicidade e alguns exemplos. Por fim, retomamos o estudo dos jogos diferenciais à luz das soluções viscosas da equação de Hamilton-Jacobi e, assim, expomos uma demonstração de existência da função de valor e do princípio da programação dinâmica a partir das noções da viscosidade / This dissertation aims to address the topic of Differential Game Theory in its connection with the Hamilton-Jacobi (HJ) equations framework. Firstly we introduce the idea of solution for a game, through the discussion of Nash equilibria and its refinements. Secondly, the solution concept is then translated to the context of Differential Games and the idea of value function is introduced in its Isaacs\'s as well as Friedman\'s version. As the value function is discussed, its relationship with the Hamilton-Jacobi equations theory becomes self-evident. Due to such relation, we investigate the HJ equation from two distinct points of view. First of all, we discuss a statement according to which if a differential game has a continuously differentiable value function, then such function is a classical solution of the HJ equation associated to the game. This result strongly relies on Bellman\'s Dynamic Programming Principle - and this is the reason why we devote an entire chapter to this theme. Furthermore, HJ is still at our sight from the PDE point of view. Our motivation is simple: under some lack of regularity - a value function which is continuous, but not continuously differentiable - a game may still have a value function represented as a solution of the associated HJ equation. In this case such a solution will be called a solution in the viscosity sense. We then discuss the properties of viscosity solutions as well as provide an existence and uniqueness theorem. Finally we turn our attention back to the theory of games and - through the notion of viscosity - establish the existence and uniqueness of value functions for a differential game within viscosity solution theory.

Esta dissertação aborda a teoria dos jogos diferenciais em sua estreita relação com a teoria das equações de Hamilton-Jacobi (HJ). Inicialmente, uma revisão da noção de solução em teoria dos jogos é empreendida. Discutem-se nesta ocasião as idéias de equilíbrio de Nash e alguns de seus refinamentos. Em seguida, tem lugar uma introdução à teoria dos jogos diferenciais, onde noções de solução como a função de valor de Isaacs e de Friedman são discutidas. É nesta altura do trabalho que fica evidente a conexão entre este conceito de solução e a teoria das equações de Hamilton-Jacobi. Por ocasião desta conexão, é explorada a noção de solução clássica e é exposta uma demonstração do fato de que se um jogo diferencial possuir uma função de valor pelo menos continuamente diferenciável, esta será uma solução da equação de Hamilton-Jacobi associada ao jogo. Este resultado faz uso do princípio da programação dinâmica, devido a Bellman, e cuja demonstração está presente no texto. No entanto, quando a função de valor do jogo é apenas contínua, então embora esta não seja uma solução clássica da equação HJ associada a jogo, vemos que ela será uma solução viscosa, ou solução no sentido da viscosidade - e a esta altura são discutidos os elementos e propriedades desta classe de soluções, um teorema de existência e unicidade e alguns exemplos. Por fim, retomamos o estudo dos jogos diferenciais à luz das soluções viscosas da equação de Hamilton-Jacobi e, assim, expomos uma demonstração de existência da função de valor e do princípio da programação dinâmica a partir das noções da viscosidade / This dissertation aims to address the topic of Differential Game Theory in its connection with the Hamilton-Jacobi (HJ) equations framework. Firstly we introduce the idea of solution for a game, through the discussion of Nash equilibria and its refinements. Secondly, the solution concept is then translated to the context of Differential Games and the idea of value function is introduced in its Isaacs\'s as well as Friedman\'s version. As the value function is discussed, its relationship with the Hamilton-Jacobi equations theory becomes self-evident. Due to such relation, we investigate the HJ equation from two distinct points of view. First of all, we discuss a statement according to which if a differential game has a continuously differentiable value function, then such function is a classical solution of the HJ equation associated to the game. This result strongly relies on Bellman\'s Dynamic Programming Principle - and this is the reason why we devote an entire chapter to this theme. Furthermore, HJ is still at our sight from the PDE point of view. Our motivation is simple: under some lack of regularity - a value function which is continuous, but not continuously differentiable - a game may still have a value function represented as a solution of the associated HJ equation. In this case such a solution will be called a solution in the viscosity sense. We then discuss the properties of viscosity solutions as well as provide an existence and uniqueness theorem. Finally we turn our attention back to the theory of games and - through the notion of viscosity - establish the existence and uniqueness of value functions for a differential game within viscosity solution theory.

Nous considérons dans ce travail, des problèmes d’optimisation dans des graphes de flot multi-agent. Trois types d’agents sont considérés : les agents producteurs, transporteurs et usagers et différentes variétés de topologies de réseaux sont abordées. Chaque agent transporteur contrôle la capacité d’un ensemble de routes élémentaires (arcs), ayant chacun une capacité qui peut être augmenté jusqu’à une valeur maximale moyennant un coût fixe. Les autres agents (i.e., usagers/producteurs) sont intéressés par la maximisation du flot qu’ils reçoivent. Dans ce but, ces derniers offrent une récompense aux agents transporteurs, cette récompense est proportionnelle à la valeur du flot reçu. Ce contexte multi-agent particulier est appelé jeu expansion de réseau multi-agent. La stratégie d’un agent transporteur consiste à décider de la capacité de ses arcs sachant qu’un coût supplémentaire est encouru pour toute expansion unitaire de capacité. Il reçoit en contrepartie une part de la récompense. Il est intéressé par la maximisation de son profit et se comporte en conséquence. En outre, la stratégie d’un agent producteur/usager consiste à décider de la politique de partage de sa récompense afin de maximiser le flot qu’il reçoit. Le flot total réalisé dépend finalement des stratégies de tous les agents. Dans ces jeux d’expansion de réseau multi-agent, nous nous intéressons à caractériser des stratégies stables (i.e., Equilibre de Nash) selon diverses hypothèses. En se basant sur cette caractérisation, différents cas sont définis et étudiés. L’analyse de la complexité de quelques problèmes de décision est présentée dans ce manuscrit. Nous nous intéressons particulièrement au problème de recherche d’un équilibre de Nash qui maximise la valeur du flot total circulant dans le réseau. Nous montrons que ce problème est NP-difficile au sens fort et nous montrons comment une telle stratégie peut être caractérisée par des chemins spécifiques dans des graphes résiduels. Nous proposons également un programme linéaire à variables mixtes (PLM) qui résout le problème dans le cas d’un seul agent producteur/usager et un ensemble d’agents transporteurs. Des résultats expérimentaux sont fournis pour prouver l’efficacité de notre approche. / In this work, multi-agent network flow problems are addressed. Three types of agentsare considered, namely the producer, transportation and customer agents and various network topologies are tackled. Every transportation agent controls the capacities of a set of elementary routes (arcs), each one having a capacity that can be increased up to a certain point at a given cost. The other agents (i.e., customers/producers) are interesting in maximizing their flow of products. For that aim, we assume that they offer to the transportation agents a reward that is proportional to the realized flow value. This particular multi-agent framework is referred to as a multi-agent network expansion game. The transportation agent’s strategy consists in deciding upon the capacity of its arcs, an extra-cost being incurred for any capacity expansion. It receives in return a part of the total reward. It is interested in the maximization of its profit and behaves accordingly. Beside that, the producers/customers’ strategies consist in deciding the sharing policy for their reward for maximizing their own flow of products. The total network flow value eventually depends on all agents’ strategies. We take interest in characterizing and finding particular stable strategies (i.e., Nash Equilibria) that are of interest for this game under various assumptions. Based on this characterization, several cases are defined and studied. The analysis of the complexity of some decision problems is made. We particularly focus on the problem of finding a Nash Equilibrium that maximizes the value of the total flow. We prove that this problem is NP-hard in the strong sense and show how such a strategy can be characterized considering paths in specific reduced agent-networks. We also provide a mixed integer linear programming (MILP) formulation that solves the problem in the case of a single producer/customer agent and a set of transportation agents. Computational experiments are provided to prove the effectiveness of our approach

This study investigates Game theory including economic theories related to agricultural production. Model creation simulating economical system is built upon theoretic starting points. Commodity production is simulated using game theory with searching for Nash equilibria. Commodity demand function is grounded on concept of Cournot game. System includes land trading based on sealed-bid auctions with second-price. This model is using external Gambit software which is freeware library for game theory computations. Model evaluation is located in Experiments section where model structure and truthfulness is tested.

This thesis is organized into two parts, one for my main area of research in the field of stochastic games, and the other for my contributions in the area of service systems. We first provide an abstract for my work in stochastic games. The field of stochastic games has been actively pursued over the last seven decades because of several of its important applications in oligopolistic economics. In the past, zero-sum stochastic games have been modelled and solved for Nash equilibria using the standard techniques of Markov decision processes. General-sum stochastic games on the contrary have posed difficulty as they cannot be reduced to Markov decision processes. Over the past few decades the quest for algorithms to compute Nash equilibria in general-sum stochastic games has intensified and several important algorithms such as stochastic tracing procedure [Herings and Peeters, 2004], NashQ [Hu and Wellman, 2003], FFQ [Littman, 2001], etc., and their generalised representations such as the optimization problem formulations for various reward structures [Filar and Vrieze, 1997] have been proposed. However, they suffer from either lack of generality or are intractable for even medium sized problems or both. In our venture towards algorithms for stochastic games, we start with a non-linear optimization problem and then design a simple gradient descent procedure for the same. Though this procedure gives the Nash equilibrium for a sample problem of terrain exploration, we observe that, in general, it need not be true. We characterize the necessary conditions and define KKT-N point. KKT-N points are those Karush-Kuhn-Tucker (KKT) points which corresponding to Nash equilibria. Thus, for a simple gradient based algorithm to guarantee convergence to Nash equilibrium, all KKT points of the optimization problem need to be KKT-N points, which restricts the applicability of such algorithms. We then take a step back and start looking at better characterization of those points of the optimization problem which correspond to Nash equilibria of the underlying game. As a result of this exploration, we derive two sets of necessary and sufficient conditions. The first set, KKT-SP conditions, is inspired from KKT conditions itself and is obtained by breaking down the main optimization problem into several sub-problems and then applying KKT conditions to each one of those sub-problems. The second set, SG-SP conditions, is a simplified set of conditions which characterize those Nash points more compactly. Using both KKT-SP and SG-SP conditions, we propose three algorithms, OFF-SGSP, ON-SGSP and DON-SGSP, respectively, which we show provide Nash equilibrium strategies for general-sum discounted stochastic games. Here OFF-SGSP is an off-line algorithm while ONSGSP and DON-SGSP are on-line algorithms. In particular, we believe that DON-SGSP is the first decentralized on-line algorithm for general-sum discounted stochastic games. We show that both our on-line algorithms are computationally efficient. In fact, we show that DON-SGSP is not only applicable for multi-agent scenarios but is also directly applicable for the single-agent case, i.e., MDPs (Markov Decision Processes). The second part of the thesis focuses on formulating and solving the problem of minimizing the labour-cost in service systems. We define the setting of service systems and then model the labour-cost problem as a constrained discrete parameter Markov-cost process. This Markov process is parametrized by the number of workers in various shifts and with various skill levels. With the number of workers as optimization variables, we provide a detailed formulation of a constrained optimization problem where the objective is the expected long-run averages of the single-stage labour-costs, and the main set of constraints are the expected long-run average of aggregate SLAs (Service Level Agreements). For this constrained optimization problem, we provide two stochastic optimization algorithms, SASOC-SF-N and SASOC-SF-C, which use smoothed functional approaches to estimate gradient and perform gradient descent in the aforementioned constrained optimization problem. SASOC-SF-N uses Gaussian distribution for smoothing while SASOC-SF-C uses Cauchy distribution for the same. SASOC-SF-C is the first Cauchy based smoothing algorithm which requires a fixed number (two) of simulations independent of the number of optimization variables. We show that these algorithms provide an order of magnitude better performance than existing industrial standard tool, OptQuest. We also show that SASOC-SF-C gives overall better performance.