Title: Solving problems using MCTS Author: Dominik Malý Department: Department of theoretical informatics and mathematical logic Supervisor: RNDr. Jan Hric Supervisor's e-mail address: Jan.Hric@mff.cuni.cz Abstract: MCTS (Monte Carlo Tree Search) methods are a state-of-the-art approach to the computer solution of strategic board game Go. Because of their versatility and successfulness, these techniques show great potential for all kinds of problems. This paper aims to explore the suitability of MCTS for solving different kind of problems, specifically games of one player, like Sudoku or SameGame. I've created a computer player based on MCTS, who can solve not only Sudoku and SameGame, but also other tasks of similar kind. I've experimentally examined many MCTS extensions and their eligibility for solving these games and through extensive testing I've also compared the suitability of various kinds of UCT selection fun- ctions and used heuristics. In case of SameGame I've compared my algorithm to another exi- sting one undertaking the same problem. In the end I've described what kind of problems has a MCTS-based computer player to overcome, if it is to successfully solve games of this type, and what characteristics should these problems posses to be suitable for MCTS solution. Keywords: MCTS, Go, Sudoku,...

Osteoarthritis (OA) affects nearly 27 million people alone and is the leading cause of chronic disability in the US. Current treatment methods target pain relief for OA via non-steroidal anti-inflammatory drugs (NSAIDs) since there are currently no clinically approved disease modifying osteoarthritis drugs (DMOADs) demonstrating efficacy. Despite multiple pre-clinical DMOAD experiments being conducted, the screening rate for drugs is severely limited by the lack of sensitivity of outcome measures and the time consuming process to achieve them. Our lab has previously developed and validated a technique to evaluate articular cartilage changes with micro-CT. The objective of this thesis was to examine two distinct treatment strategies for OA in a rat joint degeneration model using equilibrium partitioning of an ionic contrast agent based micro-CT (EPIC-µCT). We hypothesized that these novel therapeutics will assist in preventing degradation of cartilage after injury and will be measurable by EPIC-µCT. First, we characterized two of the commonly used rat joint degeneration models - monosodium iodoacetate injection (MIA) and transection of medial meniscus (MMT) in rats using EPIC-µCT. The MMT model was then used to test the effects of a matrix metalloproteinase inhibitor (MMPi) using EPIC-µCT to detect changes in articular cartilage. Finally, micronized dehydrated human amnion/chorion membrane (dHACM) was delivered intra-articularly in the MMT model and evaluated as a disease modifying OA drug.

Evolutionary and genetic algorithms are problem-solving methods designed according to a nature inspiration. They are used for solving hard problems that we cannot solve by any efficient specialized algorithm. The Monte Carlo method and its derivation the Monte Carlo Tree Search (MCTS) are based on sampling and are also commonly used for too complex problems, where we are dealing with enormous memory consumption and it is impossible to perform a complete searching. The goal of this thesis is to design a general problem solving method that is built from these two completely different approaches. We explain and implement the new method on one example problem: the Traveling salesman problem (TSP). Second part of this thesis contains various tests and experiments. We compare different settings and parametrizations of our method. The best performing variant is then compared with the classical evolutionary TSP solution or, for example, with greedy algorithms. Our method shows competitive results. The best results were achieved with the cooperation of our method and the classical evolutionary TSP solution. This union shows better results than any of its parts separately, which we find as a great success.

The Monte-Carlo Tree Search (MCTS) algorithm Upper Confidence bounds applied to Trees (UCT) has become extremely popular in computer games research. Because of the importance of this family of algorithms, a deeper understanding of when and how their different enhancements work is desirable. To avoid hard-to-analyze intricacies of tournament-level programs in complex games, this work focuses on a simple abstract game: Sum of Switches (SOS). In the SOS environment we measure the performance of UCT and two of popular enhancements: Score Bonus and the Rapid Action Value Estimation (RAVE) heuristic. RAVE is often a strong estimator, but there are some situations where it misleads a search. To mimic such situations, two different error models for RAVE are explored: random error and systematic bias. We introduce a new, more robust version of RAVE called RAVE-max to better cope with errors.

No description available.

With the increasing diversity of students attending university, there is a growing interest in the factors predicting academic performance. The objective of this study is to identify student endogenous factors (average Grade 12 grade, Grade 12 mathematics grade, Grade 12 English grade, gender, school quintile, home language, race, university gradepoint-average (GPA), university Accounting I grade and self-efficacy) that impact performance in the module, Corporate Governance I, and to identify any differences in self-efficacy levels of students of different genders and races completing this course. A quantitative research method was used, comprising multiple regression analysis, a Mann-Whitney U test and a Kruskal-Wallis H test. The data was obtained from a sample of students attempting Corporate Governance I in 2018 at the University of Cape Town. The findings indicate that on the first day of entering university, the average grade in the final year of high school was the most significant predictor of final performance in Corporate Governance I. GPA was the most significant predictor of performance at the point of registration for Corporate Governance I, before the student had embarked upon the course, as well as the point when the student had engaged with the course for a few weeks. Another interesting finding was that even though there were no significant statistical differences in the self-efficacy levels of students of different genders and races, females outperformed males while White students outperformed Black and Coloured students. This study adds to the existing literature on student performance in tertiary accounting programmes in South Africa with a particular focus on student endogenous factors. The findings could be useful for the University of Cape Town in determining its approach to teaching these students.

<p>In connection with Henriksdals wastewater treatment plant (WWTP) a smaller plant is situated called Sjöstadsverket where new methods for wastewater treatment are tested and evaluated in different process lines. On one of the lines experiments with enhanced biological phosphorous removal (EBPR), have been carried out to evaluate if it is a good alternative to traditional chemical phosphorus removal. This thesis evaluates the results from the experiments conducted during fall 2005.</p><p>The incoming water from Hammarby Sjöstad consists of only household wastewater, which gives a wastewater with high concentrations of nutrients and organic carbon. This is partly positive, because one of the limiting factors for effective biological phosphorus removal is the lack of easily biodegradable organic carbon. The high concentrations of ammonium and phosphorus demand a high reduction to achieve the requirements of outgoing concentration.</p><p>To optimize the process experiments with different recirculations and different aeration techniques have been made. The experiments have shown that it is possible to obtain good results, with periods of very low outgoing phosphorus concentrations below 0,15 mg/L. It was, however, hard to obtain a stable process and to keep a good sludge quality during the period.</p><p>Primary sludge was hydrolysed to increase the amount of easily degradable organic material into the biological step of the process. The hydrolyse had a high efficiency during the test period, but with big variations. The most desirable fraction of organic material, VFA, increased to approximately 1500 mg/L during the best periods. This is only 9 % of the amount VFA in the primary sedimentation, but could be increased with a higher flux.</p><p>When the bioP sludge is fermented as a part of sludge treatment, a supernatant from digester sludge with high concentrations of phosphorus is received. To avoid to return phosphor into the activated sludge process and to recycle phosphorus, experiments with phosphorus precipitation were made. By using struvite precipitation up to 92 % of the phosphorus in the supernatant was reduced and precipitated. This was 12 % of the total amount of incoming phosphourus and was achieved at a pH just over 9 in the experiment tank and a Mg:P quote of 1:1.</p> / <p>I anslutning till Henriksdal’s reningsverk ligger Sjöstadsverket som är ett mindre reningsverk där nya reningsmetoder testas och utvärderas i på ett antal olika försökslinjer. På en av linjerna har försök med biologisk fosforrening, även kallat BioP, utförts för att undersöka om det är ett alternativ till konventionell kemisk rening av fosfor. Detta arbete sammanfattar resultaten från de experiment som utförts under hösten 2005.</p><p>I Hammarby Sjöstad, som förser Sjöstadsverket med avloppsvatten, separeras dag och spillvatten, vilket bidrar till ett näringsrikt vatten med en hög halt organiskt material. Detta är delvis positivt, då en av de begränsande faktorerna för en effektiv biologisk fosforrening är bristen på lättillgängligt organiskt material. De höga halterna inkommande fosfor och kväve kräver dock en högre reningsgrad för att de stränga utsläppsmålen ska nås. För att optimera processen har försök med olika inställningar på recirkulationsgrader genomförts och försök med olika luftningsstrategier. Försöksperioden har visat att det går att få mycket bra resultat, med periodvis utgående fosfathalter under 0,15 mg/L. Det har dock varit svårt att få en stabil process och behålla en bra slamkvalitet under hela försöksperioden.</p><p>Primärslam från försedimenteringen har hydrolyserats för att öka mängden lättillgängligt organiskt material in till det biologiska reningssteget. Hydrolysen fungerade bra under hela försöksperioden, dock med stora variationer. Den mest önskvärda fraktionen lättillgängligt organiskt material, VFA, hade under de bästa perioderna ökat till ca 1500 mg/L i hydrolysen. Detta utgör approximativt 9 % av mängden VFA i försedimenteringen, men skulle kunna ökas vid ett större flöde.</p><p>Då BioP-slammet rötas som en del av slambehandlingen fås ett fosforrikt rejektvatten. För att undvika att få in fosfor till aktivslamprocessen och för att ta till vara på fosfor som näringsämne har försök med fosforåtervinning från rejektvatten gjorts. Genom att använda sig av struvitfällning kunde upp till 92 % av fosforn i rejektvattnet reduceras och fällas ut. Detta motsvarar 12 % av den totala inkommande fosforhalten och uppnåddes vid pH strax över 9 i struvitfällningstanken och med en Mg:P kvot på 1:1.</p>

General Game Playing (GGP) deals with the design of players that are able to play any discrete, deterministic, complete information games. For many games like chess, designers develop a player using a specially designed algorithm and tune all the features of the algorithm to play the game as good as possible. However, a general game player knows nothing about the game that is about to be played. When the game begins, game description is given to the players and they should analyze it and decide on the best way to play the game. In this thesis, we focus on two-player constant-sum simultaneous move games in GGP and how this class of games can be handled. Rock-paper-scissors can be considered as a typical example of a simultaneous move game. We introduce the CFR algorithm to the GGP community for the first time and show its effectiveness in playing simultaneous move games. This is the first implementation of CFR outside the poker world. We also improve the UCT algorithm, which is the state of the art in GGP, to be more robust in simultaneous move games. In addition, we analyze how UCT performs in simultaneous move games and argue that it does not converge to a Nash equilibrium. We also compare the usage of UCT and CFR in this class of games. Finally, we discuss about the importance of opponent modeling and how a model of the opponent can be exploited by using CFR.

This research focuses on developing AI agents that play arbitrary Atari 2600 console games without having any game-specific assumptions or prior knowledge. Two main approaches are considered: reinforcement learning based methods and search based methods. The RL-based methods use feature vectors generated from the game screen as well as the console RAM to learn to play a given game. The search-based methods use the emulator to simulate the consequence of actions into the future, aiming to play as well as possible by only exploring a very small fraction of the state-space. To insure the generic nature of our methods, all agents are designed and tuned using four specific games. Once the development and parameter selection is complete, the performance of the agents is evaluated on a set of 50 randomly selected games. Significant learning is reported for the RL-based methods on most games. Additionally, some instances of human-level performance is achieved by the search-based methods.

In connection with Henriksdals wastewater treatment plant (WWTP) a smaller plant is situated called Sjöstadsverket where new methods for wastewater treatment are tested and evaluated in different process lines. On one of the lines experiments with enhanced biological phosphorous removal (EBPR), have been carried out to evaluate if it is a good alternative to traditional chemical phosphorus removal. This thesis evaluates the results from the experiments conducted during fall 2005. The incoming water from Hammarby Sjöstad consists of only household wastewater, which gives a wastewater with high concentrations of nutrients and organic carbon. This is partly positive, because one of the limiting factors for effective biological phosphorus removal is the lack of easily biodegradable organic carbon. The high concentrations of ammonium and phosphorus demand a high reduction to achieve the requirements of outgoing concentration. To optimize the process experiments with different recirculations and different aeration techniques have been made. The experiments have shown that it is possible to obtain good results, with periods of very low outgoing phosphorus concentrations below 0,15 mg/L. It was, however, hard to obtain a stable process and to keep a good sludge quality during the period. Primary sludge was hydrolysed to increase the amount of easily degradable organic material into the biological step of the process. The hydrolyse had a high efficiency during the test period, but with big variations. The most desirable fraction of organic material, VFA, increased to approximately 1500 mg/L during the best periods. This is only 9 % of the amount VFA in the primary sedimentation, but could be increased with a higher flux. When the bioP sludge is fermented as a part of sludge treatment, a supernatant from digester sludge with high concentrations of phosphorus is received. To avoid to return phosphor into the activated sludge process and to recycle phosphorus, experiments with phosphorus precipitation were made. By using struvite precipitation up to 92 % of the phosphorus in the supernatant was reduced and precipitated. This was 12 % of the total amount of incoming phosphourus and was achieved at a pH just over 9 in the experiment tank and a Mg:P quote of 1:1. / I anslutning till Henriksdal’s reningsverk ligger Sjöstadsverket som är ett mindre reningsverk där nya reningsmetoder testas och utvärderas i på ett antal olika försökslinjer. På en av linjerna har försök med biologisk fosforrening, även kallat BioP, utförts för att undersöka om det är ett alternativ till konventionell kemisk rening av fosfor. Detta arbete sammanfattar resultaten från de experiment som utförts under hösten 2005. I Hammarby Sjöstad, som förser Sjöstadsverket med avloppsvatten, separeras dag och spillvatten, vilket bidrar till ett näringsrikt vatten med en hög halt organiskt material. Detta är delvis positivt, då en av de begränsande faktorerna för en effektiv biologisk fosforrening är bristen på lättillgängligt organiskt material. De höga halterna inkommande fosfor och kväve kräver dock en högre reningsgrad för att de stränga utsläppsmålen ska nås. För att optimera processen har försök med olika inställningar på recirkulationsgrader genomförts och försök med olika luftningsstrategier. Försöksperioden har visat att det går att få mycket bra resultat, med periodvis utgående fosfathalter under 0,15 mg/L. Det har dock varit svårt att få en stabil process och behålla en bra slamkvalitet under hela försöksperioden. Primärslam från försedimenteringen har hydrolyserats för att öka mängden lättillgängligt organiskt material in till det biologiska reningssteget. Hydrolysen fungerade bra under hela försöksperioden, dock med stora variationer. Den mest önskvärda fraktionen lättillgängligt organiskt material, VFA, hade under de bästa perioderna ökat till ca 1500 mg/L i hydrolysen. Detta utgör approximativt 9 % av mängden VFA i försedimenteringen, men skulle kunna ökas vid ett större flöde. Då BioP-slammet rötas som en del av slambehandlingen fås ett fosforrikt rejektvatten. För att undvika att få in fosfor till aktivslamprocessen och för att ta till vara på fosfor som näringsämne har försök med fosforåtervinning från rejektvatten gjorts. Genom att använda sig av struvitfällning kunde upp till 92 % av fosforn i rejektvattnet reduceras och fällas ut. Detta motsvarar 12 % av den totala inkommande fosforhalten och uppnåddes vid pH strax över 9 i struvitfällningstanken och med en Mg:P kvot på 1:1.