No description available.

No description available.

No description available.

[pt] A aprendizagem por reforço, uma técnica influente para treinar sistemas inteligentes, ganhou destaque na academia e na indústria devido à sua capacidade de resolver problemas complexos sem modelos pré-existentes. No entanto, sua aplicação a sistemas do mundo real é desafiadora devido à sua complexidade causada por altas não linearidades, amostras limitadas e restrições. Consequentemente, a pesquisa nessa área tem se concentrado principalmente em simulação, onde os modelos podem ser facilmente testados e refinados. Neste trabalho, foi proposta uma estratégia de aprendizagem por reforço para um robô bípede do mundo real aprender o comportamento de caminhada do zero. Também é apresentado um desenho de sistema focado na redução de estresse e simplicidade, garantindo um desempenho robusto, incluindo uma placa de circuito impresso personalizada para o manuseio eficiente dos componentes elétricos. O software do sistema é dividido entre a placa do sistema mestre e o sistema baseado em ROS, permitindo a comunicação entre os componentes e resolvendo o problema de perda de dados e atraso na comunicação. A simulação do modelo do robô é desenvolvida na plataforma Mujoco, incorporando propriedades físicas e parâmetros ambientais. Os algoritmos DDPG, TD3 e SAC foram utilizados para aprendizado e avaliação da técnica de destilação de política para transferência de conhecimento para uma rede mais eficiente. Finalmente, foi avaliada a transferência do aprendizado para o mundo real apresentando um experimento preliminar de aprendizado do zero no mundo real. Os resultados demonstram a eficácia do projeto do sistema robótico e dos algoritmos de aprendizado, alcançando uma caminhada estável na simulação e um máximo de catorze passos na vida real com a destilação de política do algoritmo SAC. / [en] Reinforcement learning, an influential technique for training intelligent systems, has gained prominence in academia and industry due to its ability to solve complex problems without pre-existing models. However, its application to real-world systems is challenging due to its complexity caused by high non-linearities, limited samples, and constraints. Consequently, research in this area has mainly focused on simulation, where models can be easily tested and refined. In this work, a reinforcement learning strategy towards a real-world bipedal robot to learn walking behavior from scratch was proposed. We present a robot system design focused on stress reduction and simplicity, ensuring robust performance, including a custom printed circuit board (PCB) for efficient handling of electrical components. The system s software is divided into the master system board and the ROS system, allowing communication between components and addressing data loss and communication delay issues. The robot model simulation is developed on the Mujoco platform, incorporating physical properties and environmental parameters. We utilize Deep Deterministic Policy Gradient (DDPG), Twin-Delayed Deep Deterministic Policy Gradient (TD3), and Soft Actor-Critic (SAC) algorithms for learning and evaluating the policy distillation technique for transferring knowledge to a more efficient network. Finally, we evaluate the transfer of learning to the real world and present a preliminary experiment of learning from scratch in the real world. Our results demonstrate the effectiveness of the robotic system design and the learning algorithms, achieving stable walking in simulation and a maximum of fourteen steps in real life with the policy distillation of the SAC algorithm.

The vehicle routing problem is an optimisation problem with a high computational complexity that can be solved using heuristics methods to achieve near-optimal solutions in a reasonable amount of time. The work done in this study aims to compare the execution time and distance of different routing engines when using VROOM, as well as evaluate different implementations of the k-means algorithm by looking at the rand- and adjusted rand index. The results show a difference in the distance and execution time depending on which routing engine is used and it is unclear if there is a difference in the k-means implementations. Investigating the cause behind the observed results would be interesting in future works.

Braess' paradox is a counter-intuitive phenomenon which can occur in congesting networks. It refers to those cases where the introduction of a new link in the network results in the total travel time on the network increasing. The dissertation starts by introducing the traffic assignment problem and the concept of equilibrium in traffic assignment. The concept of equilibrium is based on Wardrop's first principle that all travellers will attempt to minimize their own travel time regardless of the effect on others. A literature review includes details of a number of papers that have been published investigating theoretical aspects of the paradox. There is also a brief description of Game Theory and the Nash Equilibrium. It has been shown that the equilibrium assignment is an example of Nash Equilibrium. The majority of work that has been published deals with networks where the delay functions that are used to compute the travel times on the links of the network do not include explicit representation of the capacity of the links. In this dissertation a network that is similar in form to the one first presented by Braess was constructed with the difference being that the well-known BPR function was used in the delay functions. This network was used to show that a number of findings that had been presented previously using simpler functions also applied to this network. It was shown that when it occurs, Braess' paradox only occurs over a range of values at relatively low levels of congestion. Real-world networks were then investigated and it was found that similar results occurred to those found in the simpler test networks that are often used in discussions of the paradox. Two methodologies of eliminating the paradox were investigated and the results are presented. / Decision Sciences / M.Sc.

Depuis une vingtaine d’années, la littérature scientifique traitant de la capture attentionnelle a mis en évidence, à l’aide de paradigmes expérimentaux testés en laboratoire, un certain nombre de processus attentionnels fondamentaux. Bien que les résultats obtenus « en laboratoire » à partir de stimuli visuellement « simples » méritent encore d’être investigués, depuis quelques années se pose la question de la « transposabilité » de ces observations à des objets et des situations issus du monde réel. Les phénomènes observés en laboratoire à partir d’un matériel visuellement simple sont-ils transposables à des situations, complexes, de la vie quotidienne ? Afin de répondre à cette question, nous avons créé et testé différentes conditions proches de celles du monde réel,notamment en expérimentant en situation de double-tâche, en utilisant un matériel visuellement complexe extrait du monde réel (c.-à-d. des photographies de conduite automobile), en faisant varier la fréquence d’apparition de l’élément distracteur ou bien encore en testant une nouvelle caractéristique du distracteur, la dimension sémantique. Nos résultats révèlent les conditions d’apparition du phénomène de capture attentionnelle dans des situations s’approchant de celles du monde réel. Nous avons tout particulièrement détaillé dans cet ouvrage le cas de l’effet de capture attentionnelle contingente, phénomène majeur et robuste de la capture attentionnelle, y compris dans des situations visuellement complexes. / During the last two decades, studies about attentional capture revealed some major basic attentional processes by using several experimental paradigms. While further investigations need to be conducted by using simple visual stimuli, a raising question concerns the possibility to generalize laboratory findings to much more complex real-world situations. Indeed, basic attentional capture studies usually use simple stimuli while real-world displays are generally rich in visual information. To answer this issue, we conducted several experiments under close to real-world conditions, such as testingdual task situations, using complex visual stimuli from real-world situations (e.g. driving-scenes photographs), modulating the distractor frequency or testing attentional capture at a semantic and conceptual dimension. Our results revealed the conditions in which the attentional capture phenomenon occurs in close to real-world situations. We particularly discussed in our work the contingent attentional capture phenomenon which appears to be a strong and robust effect, in both laboratory and close to real-world situations.

Conduire un véhicule dans le brouillard requiert d’intégrer de l’information visuelle bruitée avec des attentes sur la scène routière pour rechercher des indices visuels importants pour la navigation. Les tâches d’identification et de recherche visuelle peuvent être vues comme des processus de prise de décision où l’information est accumulée et où des attentes sur l’objet et son contexte sont intégrées. L’accumulation d’information est souvent modélisée comme un processus passif. Cette thèse vise à mettre en avant des mécanismes actifs, intégrant les attentes sur la cible (sur sa position, sur son identité) et de l’information sensorielle dégradée (e.g., brouillard). Nous avons employé le paradigme de mouse-tracking, permettant d’inférer des aspects dynamiques du processus de prise de décision via les mouvements de la souris d’ordinateur. L’Étude 1 évalue l’effet du contexte dans la catégorisation de cible et suggère un compromis entre rapidité et exactitude de l’accumulation d’évidence pouvant être vu comme influençant activement la décision. Mais elle n’évalue pas directement la collecte active d’évidence. Les Études 2 et 3 incluent la mesure de la détection et vérification de la cible via les mouvement des yeux lors de la recherche visuelle dans des scènes dégradée. Les attentes sur la localisation (Étude 2) et sur l’identité de la cible (Étude 3) sont manipulées. Ces études éclairent les contributions de la détection et de la vérification dans l’accumulation d’évidence pour la réponse cible absente et cible présente. Pour conclure, nous proposons une ébauche de modèle de prise de décision intégrant une dynamique entre accumulation d’évidence et système oculomoteur. / Driving a vehicle in the fog requires the integration of noisy visual information with expectations about the visual road scene, in order to search for visual clues important for navigating. The visual search and identification of relevant objects can be seen as decision-making processes where sensory information is accumulated and where the expectations about the target object and its context are integrated. The accumulation of information is often modelled as a passive process. This thesis focuses on the contribution of active mechanisms integrating expectations about the target (its identity, its location) with degraded sensory information (with fog or artificial noise). We used the mouse-tracking paradigm, allowing to infer dynamic aspects of the decision-making process through a computer mouse movements. Study 1 evaluates the effect of the context for categorizing a target and suggests a trade-off between the speed and accuracy of the evidence accumulation process which can be seen as actively influencing the decision. But this study cannot directly evaluate the active collection of evidence. In Studies 2 and 3, target detection and verification are directly measured through eye movements during visual search tasks in visually degraded scenes. We manipulated the expectations about the location (Study 2) and the target’s identity (Study 3). These studies emphasize the contributions of the detection and verification processes in the accumulation of evidence toward the target present and target absent responses. In conclusion, we propose the draft of a decision-making model which integrates the dynamics between the accumulation of evidence, and the oculomotor system.

<p>A <i>wireless ad hoc network </i>consists of a number of mobile nodes that temporarily form a dynamic infrastructure-less network. New routing protocols that can adapt to the frequent topology changes induced by node mobility and varying link qualities are needed. During the last decade dozens of different ad hoc routing protocols have been proposed, optimized and partially compared, mainly through simulation studies.</p><p>This thesis takes an experimental approach to the evaluation of ad hoc routing protocols. We argue that real world experiments are needed in order to complement simulation studies, and to gain practical experience and insights that can provide feedback to routing protocol design and existing simulation models. For example, we discovered a performance discrepancy for the AODV protocol between real world experiments and corresponding simulation studies. This so called ``communication gray zone'' problem was explored and countermeasures were implemented. As a result we could eliminate this performance problem to a large extent. </p><p>We have implemented a software-based testbed called APE to carry out efficient and systematic experimental evaluation of ad hoc routing protocols. Experiments with up to 37 participating ad hoc nodes have demonstrated APE's ability to scale efficiently and assess repeatability between test runs. APE is part of our methodology for test repeatability in a real world ad hoc routing protocol testbed. It addresses the repeatability issue induced by stochastic factors like the radio environment and node mobility. Using APE, we have performed a systematic experimental evaluation of three ad hoc routing protocols (AODV, OLSR and LUNAR). Our results show that TCP does not work satisfactorily even in very small networks with limited mobility.</p>

This thesis explores economic aspects of virtual worlds by focusing on a specific massively multiplayer online role-playing game, RuneScape. In particular, it examines the similarities and differences between the virtual economics and real world economics, the economic understanding of RuneScape players and the possibility of using virtual worlds as a laboratory for testing economic behavior and theory. This thesis uses a versatile methodology that includes texts, direct observation, self-reports, and other reports to investigate the research questions. Virtual economics in general and RuneScape in specific are understudied so far and this study fills a gap in the literature. The unique contributions of this thesis are: a comprehensive survey that reveals player perceptions of economics, a new equation useful for modeling money supply, and a new use of faucet-drain economy in massively multiplayer online games. The results indicate that virtual economics of RuneScape partially reflects real world economics, player perceptions of virtual and real world economy are surprisingly complex, and virtual worlds can be used to study real world economics. Game developers, players, economists, educators, researchers, and individuals who are interested in massively multiplayer online games and economy in general can benefit from this study.