Deploying unmanned aerial vehicles (UAVs) as aerial base stations (BSs) in order to assist terrestrial connectivity, has drawn significant attention in recent years. UAV-BSs can take over quickly as service providers during natural disasters and many other emergency situations when ground BSs fail in an unanticipated manner. UAV-BSs can also provide cost-effective Internet connection to users who are out of infrastructure. UAV-BSs benefit from their mobility nature that enables them to change their 3D locations if the demand of ground users changes. In order to effectively make use of the mobility of UAV-BSs in a dynamic network and maximize the performance, the 3D location of UAV-BSs should be continuously optimized. However, solving the location optimization problem of UAV-BSs is NP-hard with no optimal solution in polynomial time for which near optimal solutions have to be exploited. Besides the conventional solutions, i.e. heuristic solutions, machine learning (ML), specifically reinforcement learning (RL), has emerged as a promising solution for tackling the positioning problem of UAV-BSs. The common practice for optimizing the 3D location of UAV-BSs using RL algorithms is assuming fixed location of ground users (i.e., UEs) in addition to defining discrete and limited action space for the agent of RL. In this thesis, we focus on improving the location optimization of UAV-BSs in two ways: 1-Taking into account the mobility of users in the design of RL algorithm, 2-Extending the action space of RL to a continuous action space so that the UAV-BS agent can flexibly change its location by any distance (limited by the maximum speed of UAV-BS). Three types of RL algorithms, i.e. Q-learning (QL), deep Q-learning (DQL) and actor-critic deep Q-learning (ACDQL) have been employed in this thesis to step-by-step improve the performance results of a UAV-assisted cellular network. QL is the first type of RL algorithm we use for the autonomous movement of the UAV-BS in the presence of mobile users. As a solution to the limitations of QL, we next propose a DQL-based strategy for the location optimization of the UAV-BS which largely improves the performance results of the network compared to the QL-based model. Third, we propose an ACDQL-based solution for autonomously moving the UAV-BS in a continuous action space wherein the performance results significantly outperforms both QL and DQL strategies.

Uninterrupted network connectivity is vital for real-time and mission-critical communication networks. The failure of Base Stations due to unforeseen circumstances such as natural disasters or emergencies can affect the coverage and capacity provided by terrestrial communication networks. The use of Unmanned Aerial Vehicles (UAVs) or drones in cellular networks is an upcoming area of research interest in 5G where the public sector and the communication service providers are fervently discussing it. The drones can be rapidly deployed to bridge the gaps in coverage or capacity of the network due to unforeseen circumstances. This thesis explores drone base stations' use for a simple hexagonal cell deployment scenario where the deployable base stations replace two failed macro base stations to improve the mean network capacity. Simulations show that the introduction of the deployable base stations indeed helps improve mean network capacity in case of one or multiple macro base station fail. The Genetic Algorithm is used to achieve Pareto optimality between downlink and uplink capacity of the simulated network. The simulation results show that introducing deployable nodes in a network can improve the network's capacity while also giving near-optimal transmit power values. / Oavbruten nätverksanslutning är avgörande för realtids- och missionskritiska kommunikationsnätverk. Fel på basstationer på grund av oförutsedda omständigheter som naturkatastrofer eller nödsituationer kan påverka täckningen och kapaciteten som tillhandahålls av markbundna kommunikationsnätverk. Användningen av Unmanned Aerial Vehicles (UAV) eller drönare i cellulära nätverk är ett kommande område av forskningsintresse inom 5G där den offentliga sektorn och leverantörerna av kommunikationstjänster ivrigt diskuterar det. Drönarna kan snabbt sättas in för att överbrygga klyftorna i nätverkets täckning eller kapacitet på grund av oförutsedda omständigheter. Denna avhandling utforskar drönarbasstationers användning för ett enkelt scenarie för hexagonal celldistribution där de utplacerbara basstationerna ersätter två misslyckade makrobasstationer för att förbättra den genomsnittliga nätverkskapaciteten. Simuleringar visar att introduktionen av de utplacerbara basstationerna verkligen hjälper till att förbättra den genomsnittliga nätverkskapaciteten i händelse av att en eller flera makrobasstationer misslyckas. Den genetiska algoritmen används för att uppnå Pareto-optimalitet mellan nedlänks- och upplänkkapaciteten i det simulerade nätverket. Simuleringsresultaten visar att införandet av utplacerbara noder i ett nätverk kan förbättra nätverkets kapacitet samtidigt som det ger nästan optimala värden för sändningseffekt.