Institutional Repository of Vilnius University
Reinforced learning based bot for "starcraft ii" /
Abstract
dc:descriptionThis thesis explores the development of an AI bot for the real-time strategy game StarCraft II, leveraging reinforcement learning (RL) techniques to address the complexities of dynamic, large scale environments. Specifically, the project focuses on creating a bot tailored for the Terran race, designed to compete efficiently against the game’s built-in AI opponents. Using Proximal Policy Optimization (PPO), a state-of-the-art RL algorithm, this research demonstrates the potential of AI in adapting to intricate and rapidly evolving challenges inherent in real-time strategy games. StarCraft II serves as an ideal benchmark for AI research due to its high decision-making complexity, resource management requirements, and partially observable environment. Unlike simpler games such as chess or Go, where the rules are static and decision trees finite, StarCraft II encompasses real-time decision-making under constraints, requiring both long-term strategic planning and immediate tactical execution. These characteristics make the game an excellent platform for testing advanced AI capabilities. To facilitate this project, a custom environment, Sc2Env, was developed, ensuring compati bility with Gym, a widely used reinforcement learning interface. Sc2Env encapsulates the mul tifaceted aspects of StarCraft II through multi-modal observation spaces—combining simplified visual maps with scalar values—and action spaces tailored to strategic military and economic op erations. The bot’s modular architecture supports tasks such as supply management, army training, and tactical decision-making, with reward functions designed to encourage resource efficiency, strategic unit development, and aggressive gameplay. The research draws inspiration from notable AI advancements, including DeepMind’s AlphaS tar and competitive bots like MicroMachine. These systems highlight the importance of combin ing machine learning with domain-specific customizations. For example, AlphaStar introduced human-like constraints, such as limiting actions-per-minute, and used multi-agent reinforcement learning to adapt to diverse strategies. Similarly, MicroMachine emphasizes micro-level control, achieving remarkable efficiency in unit management. This project builds upon such findings by integrating reinforcement learning with task-specific optimizations. Training scripts incorporated TensorBoard for real-time monitoring, complemented by cus tom callbacks for efficient checkpointing. Experiments revealed significant quantitative improve ments. Over 200,000 training steps, the bot achieved an 80% win rate, reducing episode lengths while stabilizing reward accumulation. Gameplay analysis showed the bot adeptly managing re sources, producing units, and executing attacks, though challenges remained in dynamic micro management and adapting to unexpected scenarios. Concluding this thesis, the research highlights the potential of hybrid approaches combining hardcoded logic and neural networks to address StarCraft II’s multifaceted challenges. Future work could include multi-agent systems, task-specific neural networks and curriculum learning to enhance scalability and training process. By refining these methodologies, the AI bot can serve as a stepping stone for broader applications, from multi-agent systems to real-world adaptive problem solving.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Drungilas, Gabrielius,
Subjects
dc:subject × 1Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- eng
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD229556785&prefLang=en_US