Processos de Decisão Markovianos (Markov Decision Process - MDP) modelam problemas de tomada de decisão sequencial em que as possíveis ações de um agente possuem efeitos probabilísticos sobre os estados sucessores (que podem ser definidas por matrizes de transição de estados). Programação dinâmica em tempo real (Real-time dynamic programming - RTDP), é uma técnica usada para resolver MDPs quando existe informação sobre o estado inicial. Abordagens tradicionais apresentam melhor desempenho em problemas com matrizes esparsas de transição de estados porque podem alcançar eficientemente a convergência para a política ótima, sem ter que visitar todos os estados. Porém essa vantagem pode ser perdida em problemas com matrizes densas de transição, nos quais muitos estados podem ser alcançados em um passo (por exemplo, problemas de controle com eventos exógenos). Uma abordagem para superar essa limitação é explorar regularidades existentes na dinâmica do domínio através de uma representação fatorada, isto é, uma representação baseada em variáveis de estado. Nesse trabalho de mestrado, propomos um novo algoritmo chamado de FactRTDP (RTDP Fatorado), e sua versão aproximada aFactRTDP (RTDP Fatorado e Aproximado), que é a primeira versão eficiente fatorada do algoritmo clássico RTDP. Também propomos outras 2 extensões desses algoritmos, o FactLRTDP e aFactLRTDP, que rotulam estados cuja função valor convergiu para o ótimo. Os resultados experimentais mostram que estes novos algoritmos convergem mais rapidamente quando executados em domínios com matrizes de transição densa e tem bom comportamento online em domínios com matrizes de transição densa com pouca dependência entre as variáveis de estado. / Markov Decision Process (MDP) model problems of sequential decision making, where the possible actions have probabilistic effects on the successor states (defined by state transition matrices). Real-time dynamic programming (RTDP), is a technique for solving MDPs when there exists information about the initial state. Traditional approaches show better performance in problems with sparse state transition matrices, because they can achieve the convergence to optimal policy efficiently, without visiting all states. But, this advantage can be lose in problems with dense state transition matrices, in which several states can be achieved in a step (for example, control problems with exogenous events). An approach to overcome this limitation is to explore regularities existing in the domain dynamics through a factored representation, i.e., a representation based on state variables. In this master thesis, we propose a new algorithm called FactRTDP (Factored RTDP), and its approximate version aFactRTDP (Approximate and Factored RTDP), that are the first factored efficient versions of the classical RTDP algorithm. We also propose two other extensions, FactLRTDP and aFactLRTDP, that label states for which the value function has converged to the optimal. The experimental results show that when these new algorithms are executed in domains with dense transition matrices, they converge faster. And they have a good online performance in domains with dense transition matrices and few dependencies among state variables.

Este trabalho é dedicado ao desenvolvimento teórico e algorítmico de processos de decisão markovianos com probabilidades imprecisas e representações relacionais. Na literatura, essa configuração tem sido importante dentro da área de planejamento em inteligência artificial, onde o uso de representações relacionais permite obter descrições compactas, e o emprego de probabilidades imprecisas resulta em formas mais gerais de incerteza. São três as principais contribuições deste trabalho. Primeiro, efetua-se uma discussão sobre os fundamentos de tomada de decisão sequencial com probabilidades imprecisas, em que evidencia-se alguns problemas ainda em aberto. Esses resultados afetam diretamente o (porém não restrito ao) modelo de interesse deste trabalho, os processos de decisão markovianos com probabilidades imprecisas. Segundo, propõe-se três algoritmos para processos de decisão markovianos com probabilidades imprecisas baseadas em programação (otimização) matemática. E terceiro, desenvolvem-se ideias propostas por Trevizan, Cozman e de Barros (2008) no uso de variantes do algoritmo Real-Time Dynamic Programming para resolução de problemas de planejamento probabilístico descritos através de versões estendidas da linguagem de descrição de domínios de planejamento (PPDDL). / This work is devoted to the theoretical and algorithmic development of Markov Decision Processes with Imprecise Probabilities and relational representations. In the literature, this configuration is important within artificial intelligence planning, where the use of relational representations allow compact representations and imprecise probabilities result in a more general form of uncertainty. There are three main contributions. First, we present a brief discussion of the foundations of decision making with imprecise probabilities, pointing towards key questions that remain unanswered. These results have direct influence upon the model discussed within this text, that is, Markov Decision Processes with Imprecise Probabilities. Second, we propose three algorithms for Markov Decision Processes with Imprecise Probabilities based on mathematical programming. And third, we develop ideas proposed by Trevizan, Cozman e de Barros (2008) on the use of variants of Real-Time Dynamic Programming to solve problems of probabilistic planning described by an extension of the Probabilistic Planning Domain Definition Language (PPDDL).

已開發國家有關海外直接投資活動的研究，自1960年代國際間海外直接投資活動如雨後春筍般出現才告展開。開發中國家的相關研究則在1980年代後期才開始，但仍屬少數。台灣地區的海外直接投資活動是從1980年代中期，投資環境惡化，生產成本上升後大量湧現。 　　本研究參酌Aharoni (1966)的投資決策階段，修正Larimo (1995)的決策過程模型，從行為學派的觀點來研究台商海外直接投資決策過程。經過八個個案的深入訪談，發現台商海外直接投資決策過程與行為學派過去結論相同者有下列四點： 　　一、投資動機為多重目的。 　　二、投資決策採可接受的原則。 　　三、實際決策時採單一去或不去的抉擇。 　　四、決策環境為動態，且非完全資訊下的理性抉擇過程。 　　而針對台灣廠商在海外直接投資過程中所表現出來的行為特色，則有下列兩點： 　　一、台商會策略性的運用合資夥伴促使投資案的順利進行。 　　二、母國政府政策在初步評估時對台商會有影響，但決策結果仍依實際效益而定。 　　在台商海外直接投資所遭遇到的跨文化管理上，本研究採Adler(1983)提出的Synergistic Research跨文化管理研究方法。實證發現，台商的跨文化管理問題多集中在人力資源管理問題上，這些問題可歸納為下列三類： 　　一、語言不同產生的溝通問題。 　　二、外派人員與當地合夥人或幹部相處合作問題。 　　三、民族性或文化差異造成政策推行的失效或差距。 　　台灣廠商在適應跨文化管理問題的過程及因應作為如下： 　　首先為了避免因與投資地主國文化不同而可能產生的管理問題，曾遭遇過跨文化管理問題的台商，會將文化差異程度列為選擇投資地主國的重要因素。 　　其次在跨文化管理作為上，海外初期營運是移植台灣母公司做法並因當地法規而略微修改。至於在當地勞工的管理上，則採「以夷制夷」方式，避免台籍幹部直接面對第一線的員工所可能因文化差異產生的管理問題。 　　在跨文化管理策略的形成上，台商目前傾向採本土化的「因地制宜」策略，並未有具體成形的「全球一致」策略。在適應過程中，台商會由文化衝突的經驗中學習，逐步修正其管理模式以配合雙方需求，並用於日後的投資案上。

<p>Idag går majoriteten av eleverna vidare från grundskolan till gymnasiet. Valalternativen är många och eleverna ställs inför ett beslut som kan vara svårt att ta. Studien handlar om gymnasievalet och beskriver skolan under 1900-talet samt karriärvägledningsteorier och socialpsykologiska teorier som kan appliceras på elevernas beslutsprocess.</p><p>Syftet är att undersöka faktorer som påverkar eleverna i deras val för att ge ökade kunskaper och insikt till studie och yrkesvägledningens profession. I en undersökning med elever från det naturvetenskapliga programmet på gymnasiet har en kvantitativ metod använts. Parallellt har även en litteraturstudie bidragit med uppgifter från tidigare forskning inom området. Resultaten från studien visar hur eleverna uppfattar faktorer som kan påverka dem.</p><p>Resultaten pekar på att eleverna anser att de väljer utifrån uppsatta mål som att gå en utbildning som ger bred grund för vidare studier och inte är så beroende av omgivningen. Studien visar också att det finns faktorer i elevens sociala sfär som påverkar dem vilket överensstämmer med tidigare forskning såsom föräldrar med högskoleutbildning eller ett samhällsklimat som idag förutsätter att ungdomar skolar sig längre än tidigare.</p> / <p>Today the majority of pupils proceed from the nine-year compulsory school to upper secondary school. There are several choices available and the pupils face a difficult decision. This study is about the choice of upper secondary school and describes the Swedish school system during the 20th century and the theories of career guidance and social psychology that can be applied to pupils’ decision-making.</p><p>The aim is to examine factors affecting the pupils in their choices to provide increased knowledge and insight into the counselling profession. A quantitative method has been used in a study with pupils from the science programme in upper secondary school. In parallel, a literature study has contributed with data from previous research in this field. The results of the study show how pupils perceive the factors that may affect them.</p><p>The results indicate that pupils make their choices on the basis of set targets like choosing an education that provides a broad basis for further studies and are not particularly affected by their social surrounding. The study also shows that there are factors in the pupil’s social sphere affecting them which is consistent with previous research, such as parents with higher education or a social climate that now requires that young people stay in education longer than before.</p><p> </p>

Idag går majoriteten av eleverna vidare från grundskolan till gymnasiet. Valalternativen är många och eleverna ställs inför ett beslut som kan vara svårt att ta. Studien handlar om gymnasievalet och beskriver skolan under 1900-talet samt karriärvägledningsteorier och socialpsykologiska teorier som kan appliceras på elevernas beslutsprocess. Syftet är att undersöka faktorer som påverkar eleverna i deras val för att ge ökade kunskaper och insikt till studie och yrkesvägledningens profession. I en undersökning med elever från det naturvetenskapliga programmet på gymnasiet har en kvantitativ metod använts. Parallellt har även en litteraturstudie bidragit med uppgifter från tidigare forskning inom området. Resultaten från studien visar hur eleverna uppfattar faktorer som kan påverka dem. Resultaten pekar på att eleverna anser att de väljer utifrån uppsatta mål som att gå en utbildning som ger bred grund för vidare studier och inte är så beroende av omgivningen. Studien visar också att det finns faktorer i elevens sociala sfär som påverkar dem vilket överensstämmer med tidigare forskning såsom föräldrar med högskoleutbildning eller ett samhällsklimat som idag förutsätter att ungdomar skolar sig längre än tidigare. / Today the majority of pupils proceed from the nine-year compulsory school to upper secondary school. There are several choices available and the pupils face a difficult decision. This study is about the choice of upper secondary school and describes the Swedish school system during the 20th century and the theories of career guidance and social psychology that can be applied to pupils’ decision-making. The aim is to examine factors affecting the pupils in their choices to provide increased knowledge and insight into the counselling profession. A quantitative method has been used in a study with pupils from the science programme in upper secondary school. In parallel, a literature study has contributed with data from previous research in this field. The results of the study show how pupils perceive the factors that may affect them. The results indicate that pupils make their choices on the basis of set targets like choosing an education that provides a broad basis for further studies and are not particularly affected by their social surrounding. The study also shows that there are factors in the pupil’s social sphere affecting them which is consistent with previous research, such as parents with higher education or a social climate that now requires that young people stay in education longer than before.

Planning in partially observable domains is a notoriously difficult problem. However, in many real-world scenarios, planning can be simplified by decomposing the task into a hierarchy of smaller planning problems which, can then be solved independently of one another. Several approaches, mainly dealing with fully observable domains, have been proposed to optimize a plan that decomposes according to a hierarchy specified a priori. Some researchers have also proposed to discover hierarchies in fully observable domains. In this thesis, we investigate the problem of automatically discovering planning hierarchies in partially observable domains. The main advantage of discovering hierarchies is that, for a plan of a fixed size, hierarchical plans can be more expressive than non-hierarchical ones. Our solution frames the discovery and optimization of a hierarchical policy as a non-convex optimization problem. By encoding the hierarchical structure as variables of the optimization problem, we can automatically discover a hierarchy. Successfully solving the optimization problem therefore yields an optimal hierarchy and an optimal policy. We describe several techniques to solve the optimization problem. Namely, we provide results using general non-linear solvers, mixed-integer linear and non-linear solvers or a form of bounded hierarchical policy iteration. Our method is flexible enough to allow any parts of the hierarchy to be specified based on prior knowledge while letting the optimization discover the unknown parts. It can also discover hierarchical policies, including recursive policies, that are more compact (potentially infinitely fewer parameters).

Planning in partially observable domains is a notoriously difficult problem. However, in many real-world scenarios, planning can be simplified by decomposing the task into a hierarchy of smaller planning problems which, can then be solved independently of one another. Several approaches, mainly dealing with fully observable domains, have been proposed to optimize a plan that decomposes according to a hierarchy specified a priori. Some researchers have also proposed to discover hierarchies in fully observable domains. In this thesis, we investigate the problem of automatically discovering planning hierarchies in partially observable domains. The main advantage of discovering hierarchies is that, for a plan of a fixed size, hierarchical plans can be more expressive than non-hierarchical ones. Our solution frames the discovery and optimization of a hierarchical policy as a non-convex optimization problem. By encoding the hierarchical structure as variables of the optimization problem, we can automatically discover a hierarchy. Successfully solving the optimization problem therefore yields an optimal hierarchy and an optimal policy. We describe several techniques to solve the optimization problem. Namely, we provide results using general non-linear solvers, mixed-integer linear and non-linear solvers or a form of bounded hierarchical policy iteration. Our method is flexible enough to allow any parts of the hierarchy to be specified based on prior knowledge while letting the optimization discover the unknown parts. It can also discover hierarchical policies, including recursive policies, that are more compact (potentially infinitely fewer parameters).

The need to analyze and closely study the gene related mechanisms motivated the research on the modeling and control of gene regulatory networks (GRN). Dierent approaches exist to model GRNs / they are mostly simulated as mathematical models that represent relationships between genes. Though it turns into a more challenging problem, we argue that partial observability would be a more natural and realistic method for handling the control of GRNs. Partial observability is a fundamental aspect of the problem / it is mostly ignored and substituted by the assumption that states of GRN are known precisely, prescribed as full observability. On the other hand, current works addressing partially observability focus on formulating algorithms for the nite horizon GRN control problem. So, in this work we explore the feasibility of realizing the problem in a partially observable setting, mainly with Partially Observable Markov Decision Processes (POMDP). We proposed a POMDP formulation for the innite horizon version of the problem. Knowing the fact that POMDP problems suer from the curse of dimensionality, we also proposed a POMDP solution method that automatically decomposes the problem by isolating dierent unrelated parts of the problem, and then solves the reduced subproblems. We also proposed a method to enrich gene expression data sets given as input to POMDP control task, because in available data sets there are thousands of genes but only tens or rarely hundreds of samples. The method is based on the idea of generating more than one model using the available data sets, and then sampling data from each of the models and nally ltering the generated samples with the help of metrics that measure compatibility, diversity and coverage of the newly generated samples.

This thesis is focused on airport departure operations; its objective is to assign a value to surface surveillance information within a collaborative framework. The research develops a cooperative concept that improves the control of departure operations at busy airports and evaluates its merit using a classical and widely accepted airport departure model. The research then assumes departure operations are collaboratively controlled and develops a stochastic model of taxi operations on the airport surface. Finally, this study investigates the effect of feeding back different levels of surface surveillance information to the departure control process. More specifically, it examines the environmental and operational impact of aircraft surface location information on the taxi clearance process. Benefits are evaluated by measuring and comparing engine emissions for given runway utilization rates.

Title from PDF of title page. / Document formatted into pages; contains 104 pages. / Thesis (M.S.I.E.)--University of South Florida, 2003. / Includes bibliographical references. / Text (Electronic thesis) in PDF format. / ABSTRACT: Medical decision problems are extremely complex owing to their dynamic nature, large number of variable factors, and the associated uncertainty. Decision support technology entered the medical field long after other areas such as the airline industry and the manufacturing industry. Yet, it is rapidly becoming an indispensable tool in medical decision making problems including the class of sequential decision problems. In these problems, physicians decide on a treatment plan that optimizes a benefit measure such as the treatment cost, and the quality of life of the patient. The last decade saw the emergence of many decision support applications in medicine. However, the existing models have limited applications to decision problems with very few states and actions. An urgent need is being felt by the medical research community to expand the applications to more complex dynamic problems with large state and action spaces. / ABSTRACT: This thesis proposes a methodology which models the class of sequential medical decision problems as a Markov decision process, and solves the model using a simulation based reinforcement learning (RL) algorithm. Such a methodology is capable of obtaining near optimal treatment strategies for problems with large state and action spaces. This methodology overcomes, to a large extent, the computational complexity of the value-iteration and policy-iteration algorithms of dynamic programming. An average reward reinforcement-learning algorithm is developed. The algorithm is applied on a sample problem of treating hereditary spherocytosis. The application demonstrates the ability of the proposed methodology to obtain effective treatment strategies for sequential medical decision problems. / System requirements: World Wide Web browser and PDF reader. / Mode of access: World Wide Web.