Um Jogo Markoviano Alternado (Alternating Markov Game - AMG) é uma extensão de um Processo de Decisão Markoviano (Markov Decision Process - MDP) para ambientes multiagentes. O modelo AMG é utilizado na tomada de decisão sequencial de n agentes quando são conhecidas as probabilidades de transição das ações a serem tomadas por cada agente. Nesse trabalho estamos interessados em AMGs com probabilidades de transição de estados imprecisas, por exemplo, quando elas são dadas na forma de intervalos de probabilidades. Apresentamos um novo modelo de AMG, que chamamos de Jogo Markoviano Alternado com Probabilidades Imprecisas (Alternate Markov Game with Imprecise Probabilities - AMGIP) que permite que as imprecisões nas probabilidades de transições de estados sejam dadas na forma de parâmetros sujeitos a restrições lineares que estende trabalhos anteriores em que a imprecisão é dada por intervalos de probabilidades (AMG-INTERVAL). Dizemos que a imprecisão representa escolhas da Natureza. A imprecisão desses modelos implica no valor do jogo ser dado por uma função intervalar. Existem diversas formas de calcular a solução do jogo, que depende do comportamento da Natureza e dos critérios de preferência dos jogadores diante das escolhas da Natureza. Assim, neste trabalho discutimos diversas soluções para o AMG-IP e AMG-INTERVAL. Também como resultado do estudo das relações existentes entre os MDPs e os AMGs, propomos um novo modelo chamado de AMG-ST (Alternating Markov Game with Set-valued Transition), capaz de modelar a incerteza do modelo MDP-ST (Markovian Decision Process with Set-valued Transition) como um jogo entre o agente e a Natureza, isto é, um jogo em que a Natureza faz o papel de um dos jogadores. / An Alternating Markov Game (AMG) is an extension of a Markov Decision Process (MDP) for multiagent environments. This model is used on sequencial decision making for n agents when we know the state transition probabilities of actions being taken by each agent. In this work we are interested in AMGs with imprecise probabilities on state transition function, for example, when they are given by probabilities intervals. We present a new AMG model, which we call Alternating Markov Game with Imprecise Probabilities (AMG-IP) that allows imprecision on state transition probabilities given by parameters subject to linear constraints that extend previous works which the imprecision is given by probabilities intervals (AMG-INTERVAL). We say that the imprecision represents the Nature choices. The imprecision of these models implies the game value is given by interval function. There are several ways to calculate the solution of the game, that depend on the behavior of the Nature and the preference criteria of the players on the choices of Nature. Therefore, in this work we discuss various solutions to AMG-IP and AMG-INTERVAL. Also from our study on the relationship among the MDPs and AMGs, we propose a new model called Alternating Markov Game with Set-valued Transition (AMG-ST), that can be used to model the uncertainty of an MDP-ST (Markovian Decision Process with Set-valued Transition) as a result of the match between the agent and the Nature, i.e., a game where the Nature is seen as one of the players.

Um Jogo Markoviano Alternado (Alternating Markov Game - AMG) é uma extensão de um Processo de Decisão Markoviano (Markov Decision Process - MDP) para ambientes multiagentes. O modelo AMG é utilizado na tomada de decisão sequencial de n agentes quando são conhecidas as probabilidades de transição das ações a serem tomadas por cada agente. Nesse trabalho estamos interessados em AMGs com probabilidades de transição de estados imprecisas, por exemplo, quando elas são dadas na forma de intervalos de probabilidades. Apresentamos um novo modelo de AMG, que chamamos de Jogo Markoviano Alternado com Probabilidades Imprecisas (Alternate Markov Game with Imprecise Probabilities - AMGIP) que permite que as imprecisões nas probabilidades de transições de estados sejam dadas na forma de parâmetros sujeitos a restrições lineares que estende trabalhos anteriores em que a imprecisão é dada por intervalos de probabilidades (AMG-INTERVAL). Dizemos que a imprecisão representa escolhas da Natureza. A imprecisão desses modelos implica no valor do jogo ser dado por uma função intervalar. Existem diversas formas de calcular a solução do jogo, que depende do comportamento da Natureza e dos critérios de preferência dos jogadores diante das escolhas da Natureza. Assim, neste trabalho discutimos diversas soluções para o AMG-IP e AMG-INTERVAL. Também como resultado do estudo das relações existentes entre os MDPs e os AMGs, propomos um novo modelo chamado de AMG-ST (Alternating Markov Game with Set-valued Transition), capaz de modelar a incerteza do modelo MDP-ST (Markovian Decision Process with Set-valued Transition) como um jogo entre o agente e a Natureza, isto é, um jogo em que a Natureza faz o papel de um dos jogadores. / An Alternating Markov Game (AMG) is an extension of a Markov Decision Process (MDP) for multiagent environments. This model is used on sequencial decision making for n agents when we know the state transition probabilities of actions being taken by each agent. In this work we are interested in AMGs with imprecise probabilities on state transition function, for example, when they are given by probabilities intervals. We present a new AMG model, which we call Alternating Markov Game with Imprecise Probabilities (AMG-IP) that allows imprecision on state transition probabilities given by parameters subject to linear constraints that extend previous works which the imprecision is given by probabilities intervals (AMG-INTERVAL). We say that the imprecision represents the Nature choices. The imprecision of these models implies the game value is given by interval function. There are several ways to calculate the solution of the game, that depend on the behavior of the Nature and the preference criteria of the players on the choices of Nature. Therefore, in this work we discuss various solutions to AMG-IP and AMG-INTERVAL. Also from our study on the relationship among the MDPs and AMGs, we propose a new model called Alternating Markov Game with Set-valued Transition (AMG-ST), that can be used to model the uncertainty of an MDP-ST (Markovian Decision Process with Set-valued Transition) as a result of the match between the agent and the Nature, i.e., a game where the Nature is seen as one of the players.

Two different sets of problems are addressed in this thesis. The first one is on partially observed semi-Markov Games (POSMG) and the second one is on semi-Markov modulated financial market model. In this thesis we study a partially observable semi-Markov game in the infinite time horizon. The study of a partially observable game (POG) involves three major steps: (i) construct an equivalent completely observable game (COG), (ii) establish the equivalence between POG and COG by showing that if COG admits an equilibrium, POG does so, (iii) study the equilibrium of COG and find the corresponding equilibrium of original partially observable problem. In case of infinite time horizon game problem there are two different payoff criteria. These are discounted payoff criterion and average payoff criterion. At first a partially observable semi-Markov decision process on general state space with discounted cost criterion is studied. An optimal policy is shown to exist by considering a Shapley’s equation for the corresponding completely observable model. Next the discounted payoff problem is studied for two-person zero-sum case. A saddle point equilibrium is shown to exist for this case. Then the variable sum game is investigated. For this case the Nash equilibrium strategy is obtained in Markov class under suitable assumption. Next the POSMG problem on countable state space is addressed for average payoff criterion. It is well known that under this criterion the game problem do not have a solution in general. To ensure a solution one needs some kind of ergodicity of the transition kernel. We find an appropriate ergodicity of partially observed model which in turn induces a geometric ergodicity to the equivalent model. Using this we establish a solution of the corresponding average payoff optimality equation (APOE). Thus the value and a saddle point equilibrium is obtained for the original partially observable model. A value iteration scheme is also developed to find out the average value of the game. Next we study the financial market model whose key parameters are modulated by semi-Markov processes. Two different problems are addressed under this market assumption. In the first one we show that this market is incomplete. In such an incomplete market we find the locally risk minimizing prices of exotic options in the Follmer Schweizer framework. In this model the stock prices are no more Markov. Generally stock price process is modeled as Markov process because otherwise one may not get a pde representation of price of a contingent claim. To overcome this difficulty we find an appropriate Markov process which includes the stock price as a component and then find its infinitesimal generator. Using Feynman-Kac formula we obtain a system of non-local partial differential equations satisfied by the option price functions in the mildsense. .Next this system is shown to have a classical solution for given initial or boundary conditions. Then this solution is used to have a F¨ollmer Schweizer decomposition of option price. Thus we obtain the locally risk minimizing prices of different options. Furthermore we obtain an integral equation satisfied by the unique solution of this system. This enable us to compute the price of a contingent claim and find the risk minimizing hedging strategy numerically. Further we develop an efficient and stable numerical method to compute the prices. Beside this work on derivative pricing, the portfolio optimization problem in semi-Markov modulated market is also studied in the thesis. We find the optimal portfolio selections by optimizing expected utility of terminal wealth. We also obtain the optimal portfolio selections under risk sensitive criterion for both finite and infinite time horizon.

No description available.