Redes de Anúncios (Ad Networks) são redes que promovem a distribuição de anúncios pela internet, de forma a maximizar o lucro total gerado pela exibição dos anúncios nos websites. Estas redes tipicamente operam através do modelo de negócios chamado CPC (Custo por Clique), em que o anunciante paga um determinado valor somente se algum usuário clicar em seu anúncio. A escolha de como o intermediador planeja a distribuição dos anúncios aos websites é de extrema importância, já que a taxa de cliques nos anúncios é extremamente baixa. Atualmente a alocação dos anúncios tem sido feita através de uma solução aproximada baseada na alocação ótima definida com dados de um período anterior, a qual é calculada através de programação linear aliada à utilização de heurísticas. Entretanto, este sistema claramente é um processo de decisão sequencial em que diversas restrições são aplicáveis, como por exemplo: o orçamento dos anunciantes, limites mínimos do número de exibições de cada anúncio, categorias dos anúncios, entre outras. Neste trabalho argumenta-se que MDPs (Markov Decision Processes) fornecem uma melhor modelagem para o problema, já que conseguem levar em conta a dinâmica do sistema, considerando, por exemplo, que um anúncio que tem poucas chances de ser clicado consiga ser alocado de forma eficiente em relação ao retorno de longo prazo, mesmo quando outros anúncios proveriam um lucro maior a curto prazo. No entanto, devido ao grande número de estados, utilizar uma solução ótima através de MDPs é impraticável. Portanto analisa-se o desempenho relativo entre o estado da arte e a modelagem ótima, obtendo garantias de que a solução aproximada baseada em programação linear não está longe da solução ótima, e que em problemas grandes (similares aos encontrados na prática) essa diferença pode ser ignorada. Por fim, propõe-se uma modelagem baseada em aprendizado por reforço para a solução deste problema, utilizando duas abordagens, uma desconsiderando informações de contexto e outra considerando informações de contexto. Aqui argumenta-se que o uso de aprendizado por reforço é mais apropriado para a solução do problema de alocação de anúncios, já que ele é capaz de adaptar sua política de alocação em função das mudanças que ocorrem como, por exemplo, no perfil do usuário. / Ad Networks promote the distribution of ads in the internet, so as to maximize the revenue generated by their display of ads in websites. These networks typically operate using the CPC (Cost per Click) business model, where the advertiser pays a monetary value when a user clicks in its advertisement. The choice of how the Ad Network distributes ads to websites is of utmost importance, since the rate of clicks on ads is extremely low. The allocation of ads has been done by an approximate solution based on data from an early period of time, which is calculated using linear programming combined with heuristics. However, this problem is clearly a sequential decision process in which multiple sequential restrictions apply, such as: the budget of the advertisers, minimum limits on the number of views for each campaign, categories of advertisements. In this dissertation we argue that MDPs (Markov Decision Processes) provide a better model for the problem, since they can automatically take into account the dynamics of the system, considering, for example, an ad with little chance of being clicked can be allocated in an efficient way, even when other ads would provide a higher profit in the short term. However, due to the large number of states, an optimal solution through MDPs is impractical; therefore we analyze here the relative performance between the linear programming and the MDP approaches, deriving guarantees that the approximate solution based on linear programming is not far from the MDP optimal solution, and in large problems (similar to those found in practice) this difference can be disregarded. Finally, we propose a model based on reinforcement learning using two different approaches, one disregarding the contextual information, and the other using contextual information. We argue that the use of reinforcement learning is more suitable for solving the problem of allocation of ads, since it is able to adapt its allocation policy to reflect changes that occur, e.g., in the user profile.

Vårt syfte med studien var att undersöka om det fanns en skillnad mellan mänoch kvinnor i deras beslutsprocess (absoluta- eller relativa process) under envittneskonfrontation, samt att kontrollera om det fanns en skillnad mellan mänoch kvinnor till ett korrekt utpekande. Våra hypoteser var att 1) kvinnor i störreutsträckning skulle använda sig av svar som beskriver en absolut beslutsprocess,2) kvinnor skulle göra fler korrekta utpekanden vid vittneskonfrontationer och 3)det fanns ett samband mellan absoluta beslutsprocessen och ett korrektutpekande. Vi genomförde en enkätundersökning som publicerades på webbensom sammanlagt besvarades av 451 personer. I enlighet med vår hypotes visaderesultaten att kvinnor använder sig av absoluta beslutsprocesser i högreutsträckning och gjorde fler korrekta utpekande än män. Resultaten diskuteradesutifrån teori och tidigare forskning. / Our purpose of the study was to investigate whether there are differences betweenmen and women in their decision-making process (absolute identification orrelative judgment) during a witness confrontation, as well as to check whetherthere are differences between men and women in identification accuracy. Ourhypotheses were: 1) women would be more likely to use answers that describe anabsolute identification, 2) women would make more correct identification inwitness confrontations, 3) there would be an association between the absoluteidentification and the correct identification. We conducted an online survey,which was answered by 451 people. In accordance with our hypothesis, theresults showed that women made use of absolute identification to a greater extentand made more correct identification than men. The results discussed based ontheory and previous research.

Redes de Anúncios (Ad Networks) são redes que promovem a distribuição de anúncios pela internet, de forma a maximizar o lucro total gerado pela exibição dos anúncios nos websites. Estas redes tipicamente operam através do modelo de negócios chamado CPC (Custo por Clique), em que o anunciante paga um determinado valor somente se algum usuário clicar em seu anúncio. A escolha de como o intermediador planeja a distribuição dos anúncios aos websites é de extrema importância, já que a taxa de cliques nos anúncios é extremamente baixa. Atualmente a alocação dos anúncios tem sido feita através de uma solução aproximada baseada na alocação ótima definida com dados de um período anterior, a qual é calculada através de programação linear aliada à utilização de heurísticas. Entretanto, este sistema claramente é um processo de decisão sequencial em que diversas restrições são aplicáveis, como por exemplo: o orçamento dos anunciantes, limites mínimos do número de exibições de cada anúncio, categorias dos anúncios, entre outras. Neste trabalho argumenta-se que MDPs (Markov Decision Processes) fornecem uma melhor modelagem para o problema, já que conseguem levar em conta a dinâmica do sistema, considerando, por exemplo, que um anúncio que tem poucas chances de ser clicado consiga ser alocado de forma eficiente em relação ao retorno de longo prazo, mesmo quando outros anúncios proveriam um lucro maior a curto prazo. No entanto, devido ao grande número de estados, utilizar uma solução ótima através de MDPs é impraticável. Portanto analisa-se o desempenho relativo entre o estado da arte e a modelagem ótima, obtendo garantias de que a solução aproximada baseada em programação linear não está longe da solução ótima, e que em problemas grandes (similares aos encontrados na prática) essa diferença pode ser ignorada. Por fim, propõe-se uma modelagem baseada em aprendizado por reforço para a solução deste problema, utilizando duas abordagens, uma desconsiderando informações de contexto e outra considerando informações de contexto. Aqui argumenta-se que o uso de aprendizado por reforço é mais apropriado para a solução do problema de alocação de anúncios, já que ele é capaz de adaptar sua política de alocação em função das mudanças que ocorrem como, por exemplo, no perfil do usuário. / Ad Networks promote the distribution of ads in the internet, so as to maximize the revenue generated by their display of ads in websites. These networks typically operate using the CPC (Cost per Click) business model, where the advertiser pays a monetary value when a user clicks in its advertisement. The choice of how the Ad Network distributes ads to websites is of utmost importance, since the rate of clicks on ads is extremely low. The allocation of ads has been done by an approximate solution based on data from an early period of time, which is calculated using linear programming combined with heuristics. However, this problem is clearly a sequential decision process in which multiple sequential restrictions apply, such as: the budget of the advertisers, minimum limits on the number of views for each campaign, categories of advertisements. In this dissertation we argue that MDPs (Markov Decision Processes) provide a better model for the problem, since they can automatically take into account the dynamics of the system, considering, for example, an ad with little chance of being clicked can be allocated in an efficient way, even when other ads would provide a higher profit in the short term. However, due to the large number of states, an optimal solution through MDPs is impractical; therefore we analyze here the relative performance between the linear programming and the MDP approaches, deriving guarantees that the approximate solution based on linear programming is not far from the MDP optimal solution, and in large problems (similar to those found in practice) this difference can be disregarded. Finally, we propose a model based on reinforcement learning using two different approaches, one disregarding the contextual information, and the other using contextual information. We argue that the use of reinforcement learning is more suitable for solving the problem of allocation of ads, since it is able to adapt its allocation policy to reflect changes that occur, e.g., in the user profile.

<p>The present licentiate thesis deals with decision processes, entrepreneurship, and politics in a local context. The study focuses on the local project <em>Björntanden</em> in Östersund, an entrepreneurial idea aiming at regional development. The purpose of the thesis is to understand the decision processes that enable or hinder the realization of entrepreneurial ideas in a local context.</p><p>The background of the study is the notion that regional development often is considered a condition for economic growth. Concepts such as local and entrepreneurial processes are commonly used in this context. These processes are seldom run in isolation by single actors but normally in cooperation with others. It is not uncommon that actors such as politicians and other representatives of local government and state agencies play an important role in influencing the conditions for local entrepreneurship. As decision processes in the private and public sector are different, this creates coordination problems.</p><p>Cooperation between many actors also tends to create coordination problems when actors with different organisational principles and organisational cultures meet in a joint arena. This is further accentuated when business people, civil servants, politicians and others are to cooperate in decision-making on entrepreneurial ideas that often are inherently unclear.</p><p>I followed the project <em>Björntanden</em> for four years by observations, interviews and studies of published and unpublished documents. On the basis of the experiences of the actors involved I have interpreted the meaning of their actions, i.e., what they have said and what they have done, in order to form the concepts used in the study. Through an interactive process between received theory and the gradually evolving results of the empirical study I have attempted to reach an understanding of the decision processes by linking local conditions, i.e. points of departure for entrepreneurial ideas in a local context, with different types of decision processes. The analysis indicates aspects that may create possibilities and space as well as obstacles for successful decision-making when entrepreneurship and politics are to act in cooperation.</p> / <p>I den här licentiatuppsatsen behandlar jag beslutsprocesser, entreprenörskap och politik i en lokal kontext. Studien fokuserar på det lokala projektet Björntanden i Östersund, en entreprenöriell idé för regional utveckling. Syftet med uppsatsen är att förstå de beslutsprocesser som ger möjlighet eller hindrar att entreprenöriella idéer i en lokal kontext förverkligas.</p><p>Bakgrunden till studien är att regional utveckling ofta framhålls som en förutsättning för ekonomisk tillväxt. Begrepp som lokala och entreprenöriella processer är vanligt förekommande i ett sådant sammanhang. Dessa processer bedrivs sällan isolerade hos någon enskild aktör utan ofta i samverkan med andra. Vidare är det inte ovanligt att aktörer såsom politiker, kommunala och statliga företrädare har en framträdande roll för de möjligheter och hinder som gäller för entreprenörskap i ett lokalt sammanhang. Då beslutsprocesser inom den privata sektorn skiljer sig från beslutsprocesser inom den offentliga innebär det koordinationsproblem.</p><p>Samverkan mellan många aktörer tenderar också att leda till koordinationsproblem när mötet mellan aktörer med olika organisationsprinciper och organisationskulturer skall hanteras på en gemensam arena. Detta blir än mer framträdande, när näringslivsföreträdare, offentliga tjänstemän, politiker och andra skall samverka kring beslut om entreprenöriella idéer, vars signum många gånger är det otydliga.</p><p>Under fyra år har jag följt projektet genom observationer, enskilda samtal och studier av publicerade och opublicerade handlingar. Jag har utgått från aktörernas egna upplevelser och erfarenheter av projektet Björntanden och tolkat möjliga innebörder i deras ageranden i form av vad de har sagt och vad de har gjort som underlag för begreppsbildningen i studien. Genom växelspel mellan teoretiska utgångspunkter och den empiriska undersökningens gradvis framväxande resultat har jag försökt uppnå förståelse av beslutsprocesserna genom att sammankoppla lokala förutsättningar, alltså aktuella <em>points of departure</em> för entreprenöriella idéer, i en lokal kontext, med olika slag av beslutsprocesser. Analysen visar på aspekter som kan skapa möjligheter och ge utrymme men också hinder för framgångsrikt beslutsfattande när entreprenörskap och politik samverkar.</p>

The present licentiate thesis deals with decision processes, entrepreneurship, and politics in a local context. The study focuses on the local project Björntanden in Östersund, an entrepreneurial idea aiming at regional development. The purpose of the thesis is to understand the decision processes that enable or hinder the realization of entrepreneurial ideas in a local context. The background of the study is the notion that regional development often is considered a condition for economic growth. Concepts such as local and entrepreneurial processes are commonly used in this context. These processes are seldom run in isolation by single actors but normally in cooperation with others. It is not uncommon that actors such as politicians and other representatives of local government and state agencies play an important role in influencing the conditions for local entrepreneurship. As decision processes in the private and public sector are different, this creates coordination problems. Cooperation between many actors also tends to create coordination problems when actors with different organisational principles and organisational cultures meet in a joint arena. This is further accentuated when business people, civil servants, politicians and others are to cooperate in decision-making on entrepreneurial ideas that often are inherently unclear. I followed the project Björntanden for four years by observations, interviews and studies of published and unpublished documents. On the basis of the experiences of the actors involved I have interpreted the meaning of their actions, i.e., what they have said and what they have done, in order to form the concepts used in the study. Through an interactive process between received theory and the gradually evolving results of the empirical study I have attempted to reach an understanding of the decision processes by linking local conditions, i.e. points of departure for entrepreneurial ideas in a local context, with different types of decision processes. The analysis indicates aspects that may create possibilities and space as well as obstacles for successful decision-making when entrepreneurship and politics are to act in cooperation. / I den här licentiatuppsatsen behandlar jag beslutsprocesser, entreprenörskap och politik i en lokal kontext. Studien fokuserar på det lokala projektet Björntanden i Östersund, en entreprenöriell idé för regional utveckling. Syftet med uppsatsen är att förstå de beslutsprocesser som ger möjlighet eller hindrar att entreprenöriella idéer i en lokal kontext förverkligas. Bakgrunden till studien är att regional utveckling ofta framhålls som en förutsättning för ekonomisk tillväxt. Begrepp som lokala och entreprenöriella processer är vanligt förekommande i ett sådant sammanhang. Dessa processer bedrivs sällan isolerade hos någon enskild aktör utan ofta i samverkan med andra. Vidare är det inte ovanligt att aktörer såsom politiker, kommunala och statliga företrädare har en framträdande roll för de möjligheter och hinder som gäller för entreprenörskap i ett lokalt sammanhang. Då beslutsprocesser inom den privata sektorn skiljer sig från beslutsprocesser inom den offentliga innebär det koordinationsproblem. Samverkan mellan många aktörer tenderar också att leda till koordinationsproblem när mötet mellan aktörer med olika organisationsprinciper och organisationskulturer skall hanteras på en gemensam arena. Detta blir än mer framträdande, när näringslivsföreträdare, offentliga tjänstemän, politiker och andra skall samverka kring beslut om entreprenöriella idéer, vars signum många gånger är det otydliga. Under fyra år har jag följt projektet genom observationer, enskilda samtal och studier av publicerade och opublicerade handlingar. Jag har utgått från aktörernas egna upplevelser och erfarenheter av projektet Björntanden och tolkat möjliga innebörder i deras ageranden i form av vad de har sagt och vad de har gjort som underlag för begreppsbildningen i studien. Genom växelspel mellan teoretiska utgångspunkter och den empiriska undersökningens gradvis framväxande resultat har jag försökt uppnå förståelse av beslutsprocesserna genom att sammankoppla lokala förutsättningar, alltså aktuella points of departure för entreprenöriella idéer, i en lokal kontext, med olika slag av beslutsprocesser. Analysen visar på aspekter som kan skapa möjligheter och ge utrymme men också hinder för framgångsrikt beslutsfattande när entreprenörskap och politik samverkar.

This thesis considers the problem of performance modeling, analysis and control of capacitated re-entrant lines. Specifically, the first part of the thesis develops an analytical framework for the modeling, analysis and control of capacitated re-entrant lines, which is based on Generalized Stochastic Petri nets (GSPN) framework. The corresponding scheduling problem is systematically formulated, and the structure of the optimal policy is characterized and compared to that identified for "traditional" re-entrant lines. The second part of thesis addresses the problem of developing a systematic and computationally effective method for computing the optimal scheduling policy for any given configuration of capacitated re-entrant line. Specifically, the underlying scheduling problem is transformed to a Markov Decision Process (MDP) problem and an algorithm that systematically generates the MDP formulation for any given fab configuration is developed. The third part of thesis develops an effective approximating scheme based on the Neuro-Dynamic Programming (NDP) theory. In its general definition, the NDP method seeks the approximation of the optimal relative value function of the underlying MDP formulation by a parameterized function. Hence, an approximating structure for the considered problem is proposed and the quality of the generated approximations is systematically assessed. More specifically, this part of the thesis develops a set of "feature" functions and the mathematical apparatus necessary to evaluate the considered approximating scheme through a numerical experiment. The obtained results indicate that good quality approximations can be achieved by considering a set of features that characterize the distribution of the running process instances to the various processing stages and their lower order interactions. The last part of the thesis exploits the performance models developed in its earlier parts in order to provide an analytical characterization of the optimality of various deadlock resolution strategies for Markovian resource allocation systems under the objective of maximizing throughput.

Includes vita. / Title from PDF of title page. / Document formatted into pages; contains 111 pages. / Thesis (Ph.D.)--University of South Florida, 2003. / Includes bibliographical references. / Text (Electronic thesis) in PDF format. / ABSTRACT: A large class of sequential decision making problems under uncertainty with multiple competing decision makers/agents can be modeled as stochastic games. Stochastic games having Markov properties are called Markov games or competitive Markov decision processes. This dissertation presents an approach to solve non cooperative stochastic games, in which each decision maker makes her/his own decision independently and each has an individual payoff function. In stochastic games, the environment is nonstationary and each agent's payoff is affected by joint decisions of all agents, which results in the conflict of interest among the decision makers. In this research, the theory of Markov decision processes (MDPs) is combined with the game theory to analyze the structure of Nash equilibrium for stochastic games. In particular, the Laurent series expansion technique is used to extend the results of discounted reward stochastic games to average reward stochastic games. / ABSTRACT: As a result, auxiliary matrix games are developed that have equivalent equilibrium points and values to a class of stochastic games that are irreducible and have average reward performance metric. R-learning is a well known machine learning algorithm that deals with average reward MDPs. The R-learning algorithm is extended to develop a Nash-R reinforcement learning algorithm for obtaining the equivalent auxiliary matrices. A convergence analysis of the Nash-R algorithm is developed from the study of the asymptotic behavior of its two time scale stochastic approximation scheme, and the stability of the associated ordinary differential equations (ODEs). The Nash-R learning algorithm is tested and then benchmarked with MDP based learning methods using a well known grid game. Subsequently, a real life application of stochastic games in deregulated power market is explored. / ABSTRACT: According to the current literature, Cournot, Bertrand, and Supply Function Equilibrium (SFEs) are the three primary equilibrium models that are used to evaluate the power market designs. SFE is more realistic for pool type power markets. However, for a complicated power system, the convex assumption for optimization problems is violated in most cases, which makes the problems more difficult to solve. The SFE concept in adopted in this research, and the generators' behaviors are modeled as a stochastic game instead of one shot game. The power market is considered to have features such as multi-settlement (bilateral, day-ahead market, spot markets and transmission congestion contracts), and demand elasticity. Such a market consisting of multiple competing suppliers (generators) is modeled as a competitive Markov decision processes and is studied using the Nash-R algorithm. / System requirements: World Wide Web browser and PDF reader. / Mode of access: World Wide Web.

Uncertainty is a feature of many AI applications. While there are polynomial-time algorithms for planning in stochastic systems, planning is still slow, in part because most algorithms plan for all eventualities. Algorithms such as LAO* are able to find good or optimal policies more quickly when the starting state of the system is known. In this thesis we present an extension to LAO*, called BLAO*. BLAO* is an extension of the LAO* algorithm to a bidirectional search. We show that BLAO* finds optimal or E-optimal solutions for goal-directed MDPs without necessarily evaluating the entire state space. BLAO* converges much faster than LAO* or RTDP on our benchmarks.

This thesis focuses on increasing the efficiency of systems with cross-trained workforce and finite storage spaces. Our objective is to maximize the throughput and minimize the setup costs (if they exist). More specifically, we are interested in determining effective cross-training strategies and dynamic server assignment policies for flexible servers in production lines with finite buffers. In the first part of this thesis, we study non-Markovian systems and support the conjecture that effective server assignment policies are robust to service time distributions. Next, we consider understaffed tandem lines with partially or fully flexible servers, determine optimal and heuristic server assignment policies, and show that most of the benefits of full flexibility can be achieved with limited flexibility. Finally, we incorporate the setups to our model, determine the optimal server assignment policy for some systems and show how the effective assignment of servers depends on the magnitude of the setup costs.

abstract: Today's competitive markets force companies to constantly engage in the complex task of managing their demand. In make-to-order manufacturing or service systems, the demand of a product is shaped by price and lead times, where high price and lead time quotes ensure profitability for supplier, but discourage the customers from placing orders. Low price and lead times, on the other hand, generally result in high demand, but do not necessarily ensure profitability. The price and lead time quotation problem considers the trade-off between offering high and low prices and lead times. The recent practices in make-to- order manufacturing companies reveal the importance of dynamic quotation strategies, under which the prices and lead time quotes flexibly change depending on the status of the system. In this dissertation, the objective is to model a make-to-order manufacturing system and explore various aspects of dynamic quotation strategies such as the behavior of optimal price and lead time decisions, the impact of customer preferences on optimal decisions, the benefits of employing dynamic quotation in comparison to simpler quotation strategies, and the benefits of coordinating price and lead time decisions. I first consider a manufacturer that receives demand from spot purchasers (who are quoted dynamic price and lead times), as well as from contract customers who have agree- ments with the manufacturer with fixed price and lead time terms. I analyze how customer preferences affect the optimal price and lead time decisions, the benefits of dynamic quo- tation, and the optimal mix of spot purchaser and contract customers. These analyses necessitate the computation of expected tardiness of customer orders at the moment cus- tomer enters the system. Hence, in the second part of the dissertation, I develop method- ologies to compute the expected tardiness in multi-class priority queues. For the trivial single class case, a closed formulation is obtained. For the more complex multi-class case, numerical inverse Laplace transformation algorithms are developed. In the last part of the dissertation, I model a decentralized system with two components. Marketing department determines the price quotes with the objective of maximizing revenues, and manufacturing department determines the lead time quotes to minimize lateness costs. I discuss the ben- efits of coordinating price and lead time decisions, and develop an incentivization scheme to reduce the negative impacts of lack of coordination. / Dissertation/Thesis / Ph.D. Industrial Engineering 2012