National Technical University of Athens (NTUA)
Τεχνητή νοημοσύνη για παιχνίδια: ανάπτυξη πρακτόρων με χρήση βαθιάς μάθησης
Abstract
dc:descriptionArtificial intelligence has been evolving at a rapid pace in the last decade, being part of many scientific fields such as IT, medicine and even education. Its ever-increasing application in different fields contributes on the one hand to the continuous development of modern techniques and algorithms, on the other hand leads to additional requirements and new challenges for the wider field of artificial intelligence. A field of particular interest concerns the application of such algorithms in the context of games, the environments of which offer immediate feedback and at the same time introduce several difficulties and limitations. The aim of this thesis is the development of intelligent agents for video games using artificial intelligence and machine learning techniques. Under this scope, the currently prevailing techniques are examined and new algorithms and methods are proposed in order to deal with the main appearing challenges. In the first stage of the thesis, the implementation of agents based on genetic algorithms was examined. In particular, the possibility of applying a genetic algorithm as a standalone technique for encoding the states of the environment and the final decision-making by the agent was examined. In this context, a novel method for the states' representation was designed in order to reduce the state space and make feasible the implementation of the proposed approach. The representation logic was based on N-groups of blocks in order to encode states using the least possible information. The experiments carried out highlighted the functionality of this particular technique by ranking it higher than corresponding methods of different evolutionary-based approaches. Afterwards the behavior of an intelligent agent in stochastic environments was studied. In this case, the Monte Carlo tree search algorithm was mainly studied as it is the state-of-the-art approach to a large subset of the broader field of artificial intelligence for games. The first section that was considered was the optimization of the stage in which the states which are subsequently used by the algorithm are evaluated. To that end, a gradient boosting classifier trained on a specially designed dataset, was incorporated into the algorithm significantly increasing its accuracy in estimating the states' values. In addition, a stochastic evaluation process of the tree nodes was implemented aiming to better adapt the model to the branching factor. In order to exploit information more efficiently based on the trees' depth, an early simulation technique was also applied in the early stages of the algorithm leading to stronger classifier predictions. The combination of the above methods led to a great increase of the agent's performance which surpassed the optimal algorithm provided by the used framework. Then, the optimization of the selection step of the algorithm was evaluated. In this respect, the above methodology was additionally enhanced with a novel pruning technique based on the use of artificial neural networks, in order to reduce the search space. The goal is to remove the actions with low expected value from the set of possible actions during the execution of the algorithm and utilize the remaining computing resources to more accurately evaluate the remaining actions. To implement this technique, two separate neural networks were trained and used collaboratively. By combining the outputs of the two networks, the optimal pair of iterations and number of actions to be pruned is determined and the search space is being reduced iteratively until the algorithm is finished. The networks were trained on synthetic training data which were derived from a special simulation environment implemented for this purpose. The pruning technique was tested individually as well as in conjunction with the gradient boosting classifier leading to further improvement of the algorithm's performance. In the next section, the enhancement of the selection phase of the Monte Carlo tree search algorithm without domain knowledge was examined. In this case, the core idea is to match similar nodes in order to use – during the selection phase – statistics of nodes located at a higher level of the search tree which therefore have more accurate data (as they have been visited more times during the search). In this light, two different state matching methodologies were implemented based on the sequence of actions that preceded. In the first case, the nodes' similarity was based on the length of identical N-grams from which they were derived, while in the second case it was based on a specifically designed representation of the similarity of executed actions. The proposed techniques were applied to general intelligent agents' environments achieving higher performance than relevant approaches concerning the selection phase of the algorithm in the majority of cases. In the last part of the thesis, the field of reinforcement learning was investigated. In particular, the introduction of a data augmentation technique based on generative models for the creation of new, synthetic states was examined, aiming to make the training of the agent more effective. Additionally, a model was designed to predict the action performed between two successive states in order to be able to produce complete samples in the form needed to solve reinforcement learning problems. These models are combined to create synthetic data for cases of high and low immediate rewards, which are encountered less frequently during the agent’s interaction with the environment. The proposed methodology, in which the synthetic samples are mixed with the actually observed data during the training process, was evaluated in different, heterogeneous environments achieving an increase in the agent's total obtained reward compared to traditional image augmentation techniques.
Degree
thesis:*- Grantor dc:publisher
- National Technical University of Athens (NTUA)
- Year dc:date
- 2024
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Papagiannis, Anastasios
- Παπαγιάννης, Αναστάσιος
Subjects
dc:subject × 26- Τεχνητή νοημοσύνη
- Βαθιά μηχανική μάθηση
- Ευφυείς πράκτορες
- Νευρωνικά δίκτυα
- Γενετικοί αλγόριθμοι
- Δενδρική αναζήτηση Μόντε Κάρλο
- Τεχνικές κλαδέματος
- Ενισχυτική μάθηση
- Γεννητική επαύξηση δεδομένων
- Ηλεκτρονικά παιχνίδια
- Artificial intelligence
- Deep learning
- Intelligent agents
- Neural networks
- Genetic algorithms
- Monte Carlo tree search
- Pruning techniques
- Reinforcement learning
- Generative data augmentation
- Video games
- Επιστήμη Ηλεκτρολόγου Μηχανικού, Ηλεκτρονικού Μηχανικού, Μηχανικού Η/Υ
- Επιστήμες Μηχανικού και Τεχνολογία
- Υπολογιστές, Υλικό (hardware) και Αρχιτεκτονική
- Electrical Engineering, Electronic Engineering, Information Engineering
- Engineering and Technology
- Computer science, Hardware and Architecture
Rights
- Language dc:language
- gre
Identifiers
dc:identifier.*- Identifier
- 10.12681/eadd/56731