Millimeter wave systems leverage beamforming to generate narrow, high-powered beams for overcoming the increased path loss in the millimeter wave spectrum. These beams are spa- tially confined, making millimeter wave links more resilient to eavesdropping and jamming attacks. However, the millimeter wave radios locate each other and establish communica- tion by exhaustively probing all possible angular directions, increasing their susceptibility to attacks. In this thesis, we showcase a secure beam management solution where we apply an adaptive beam management procedure that avoids probing the directions of potential attackers. We employ a reinforcement learning agent to control the probing and dynami- cally restrict sweeps to a subset of beams in the millimeter wave transmitter codebook to avoid the locations of potential attackers based on a proposed metric that quantifies the beam sweeping secrecy over a pre-defined area. We evaluate our proposed system through numerical simulations and an experimental real-life implementation on the CCI xG Testbed. / Master of Science / Millimeter wave systems leverage beamforming, a technique that's used to direct both trans- mission and reception of a signal to create narrow, high-powered beams that can overcome the signal deterioration that comes with millimeter wave spectrum. The spatially confined nature of these beams makes millimeter wave links resilient to eavesdropping and jamming attacks. However, the millimeter wave radios find each other and establish communication by searching every possible angular direction, which increases the potential for the millimeter wave radios to be attacked. In this thesis, we showcase a secure method of establishing this communication link that avoids looking in the direction of a potential attacker. We then employ an artificial intelligence capable of controlling this search by sweeping a subset of all possible directions in the millimeter wave transmitter codebook based on a proposed metric that quantifies the secrecy of communication. We evaluate our proposed system through numerical simulations and an experimental real-life implementation on the CCI xG Testbed.

Millimeter wave (mmWave) plays a critical role in the Fifth-generation (5G) new radio due to the rich bandwidth it provides. However, one shortcoming of mmWave is the substantial path loss caused by poor diffraction at high frequencies, and consequently highly directional beams are applied to mitigate this problem. A typical way of beam management is to cluster users based on their locations. However, localization uncertainty is unavoidable due to measurement accuracy, system performance fluctuation, and so on. Meanwhile, the traffic demand may change dynamically in wireless environments, which increases the complexity of network management. Therefore, a scheme that can handle both the uncertainty of localization and dynamic radio resource allocation is required. Moreover, since the localization uncertainty will influence the network performance, more state-of-the-art localization methods, such as vision-aided localization, are expected to reduce the localization error. In this thesis, we proposed two algorithms for joint radio resource allocation and beam management in 5G mmWave networks, namely UK-means-based Clustering and Deep Reinforcement Learning-based resource allocation (UK-DRL) and UK-medoids-based Clustering and Deep Reinforcement Learning-based resource allocation (UKM-DRL). Specifically, we deploy UK-means and UK-medoids clustering method in UK-DRL and UKM-DRL, respectively, which is designed to handle the clustering under location uncertainties. Meanwhile, we apply Deep Reinforcement Learning (DRL) for intra-beam radio resource allocations in UK-DRL and UKM-DRL. Moreover, to improve the localization accuracy, we develop a vision-aided localization scheme, where pixel characteristics-based features are extracted from satellite images as additional input features for location prediction. The simulations show that UK-DRL and UKM-DRL successfully improve the network performance in data rate and delay than baseline algorithms. When the traffic load is 4 Mbps, UK-DRL has a 172.4\% improvement in sum rate and 64.1\% improvement in latency than K-means-based Clustering and Deep Reinforcement Learning-based resource allocation (K-DRL). UKM-DRL has 17.2\% higher throughput and 7.7\% lower latency than UK-DRL, and 231\% higher throughput and 55.8\% lower latency than K-DRL. On the other hand, the vision-aided localization scheme can significantly reduce the localization error from 17.11 meters to 3.6 meters.

The growing adoption of millimeter wave (mmWave) turns efficient beamforming and beam management procedures into key enablers for 5th Generation (5G) and Beyond 5G (B5G) mobile networks. Recent research has sought to optimize beam management in modern Radio Access Network (RAN) architectures, where open, virtualized, disaggregated and multi-vendor environments are considered, and management platforms allow the use of Artificial Intelligence (AI) and Machine Learning (ML)-based solutions. Moreover, beam management represents some fundamental use cases defined by Open RAN Alliance (O-RAN). This work analyses beam management strategies in Open RAN and proposes solutions for codebook-based mmWave systems inspired by two use cases from O-RAN: the Grid of Beams (GoB) Optimization and the AI/ML-assisted Beam Selection. For the GoB Optimization use case, a scenario subject to constraints on the use of the full GoB due to overhead during beam selection is considered. An Advantage Actor Critic (A2C) learning-based framework is proposed to optimize the GoB, as well as the transmission power in a mmWave network. The proposed technique improves Energy Efficiency (EE) and ensures fair coverage is maintained. The simulations show that A2C-based joint optimization of GoB and transmission power is more effective than using Equally Spaced Beams (ESB) and fixed power, or the optimization of GoB and transmission power disjointly. Compared to the ESB and fixed transmission power strategy, the proposed approach achieves more than twice the average EE in the scenarios under test, and it is closer to the maximum theoretical EE. In the case of the AI/ML-assisted Beam Selection use case, the overhead during beam selection is addressed by a multi-modal sensing-aided ML-based method. When using sensing information sources external to the RAN in a multi-vendor disaggregated environment, such methods must account for privacy and data ownership issues. A Distributed Machine Learning (DML) strategy based on Split Learning (SL) is proposed to this end. The solution can cope with deployment challenges in novel RAN architectures and is applied to single and multi-level beam selection decisions, where the latter considers hierarchical codebook structures. With the proposed approach, accuracy levels above 90% can be achieved, while overhead decreases by 85% or more. SL achieves performance comparable to the centralized learning-based strategies, with the added value of accounting for privacy and data ownership issues.

[ES] La llegada de la próxima generación del estándar de comunicaciones móviles, la llamada quinta generación (5G), es prácticamente una realidad. Las primeras redes comerciales han comenzado a ser desplegadas, centrándose en ofrecer altas velocidades de transferencia de datos. Sin embargo, el estándar 5G va mucho más allá y prevé dar soporte a nuevos servicios que pretenden revolucionar la sociedad. Estos nuevos servicios imponen un nivel alto de requisitos en no solo en cuanto a velocidad del tráfico de datos, sino en cuanto a latencia o número de dispositivos conectados simultáneamente. La amplia variedad de requisitos no puede ser soportada por las redes de cuarta generación (4G), por lo que se hizo necesario plantear un nuevo paradigma para las redes inalámbricas. Con la promesa de grandes cantidades de ancho de banda sin utilizar, el estándar 5G contempla utilizar frecuencias en la comúnmente conocida como banda de milimétricas (mmWave). Esta banda presenta grandes pérdidas de propagación, que se acentúan si existen bloqueos de señal. Actividades regulatorias del uso de las bandas de milimétricas atrajo el interés tanto de la industria como de la academia en plantear soluciones para dar servicio en estas bandas. En los últimos años se han presentado infinidad de trabajos basados en sistemas con múltiples antenas o MIMO, para conformar las señales transmitidas o recibidas en haces apuntando en determinadas direcciones. La ganancia que aportan los sistemas MIMO pueden compensar las altas pérdidas de propagación, asegurando la viabilidad de las comunicaciones mmWave. Se ha detectado una evidente falta de estudios sobre la viabilidad de sistemas MIMO en entornos móviles y dinámicos con bloqueos que hagan necesario que el sistema se reconfigure. Esta Tesis pretende cubrir este espacio desde un enfoque práctico y propone mecanismos de gestión de los haces para hacerles un seguimiento utilizando los recursos y mecanismos del nuevo estándar 5G. Las soluciones aportadas se basan en el uso eficiente de los reportes de medidas de las señales de referencia estandarizadas en enlace descendente. En primer lugar, esta Tesis recoge un análisis minucioso del estado del arte, donde se corrobora la necesidad de aportar soluciones de seguimiento de haces en sistemas de comunicaciones en la banda de milimétricas. Además, se estudian los diferentes mecanismos definidos en el estándar 5G y que posibilitan el seguimiento. Cabe destacar que el estándar no define un mecanismo único a seguir, permitiendo presentar propuestas. Una vez conocidas las tecnologías, se centra el estudio en el impacto del seguimiento sobre las prestaciones a nivel de red y de enlace. Dicho estudio se realiza sobre un sistema punto a punto, donde el terminal móvil se desplaza por un entorno urbano. En base a simulaciones de red, se cuantifica el índice de seguimiento de haz y de cómo dicho seguimiento afecta a la relación señal a ruido más interferencia (SINR) y la tasa de transmisión del usuario. Las soluciones de seguimiento propuestas en esta Tesis se pueden clasificar en dos categorías. En una primera categoría, se realiza el seguimiento en base a reportes de medidas de las señales de referencia. Independientemente de la velocidad, se alcanza un seguimiento del 91% con poca penalización en la tasa de transmisión si se monitorizan los haces de interés con una periodicidad menor de 20 ms. En la segunda categoría caben mecanismos de seguimiento que hacen uso de fuentes externas de información. Dentro de esta categoría, se propone un fingerprinting que relacione haces con la localización reportada y un modelo de machine learning (ML) que prediga los haces a utilizar. El fingerprinting proporciona los mismos niveles de rendimiento. Sin embargo, esta solución es muy sensible a errores y requiere considerar todos los casos posibles, lo que la hace tecnológicamente inviable. En cambio, el modelo de ML, que hace p / [CA] L'arribada de la següent generació de l'estàndard de comunicacions mòbils, l'anomenada cinquena generació (5G), es pràcticament una realitat. Les primeres xarxes comercials han començat a desplegar-se i s'han centrat en oferir altes velocitats de transferència de dades. No obstant, l'estàndard 5G va molt mes allà y preveu donar suport a nous serveis que pretenen revolucionar la societat. Estos nous serveis imposen un alt nivell de requisits no sols en quant a velocitat de tràfic de dades, si no també en quant a latència o número de connexions simultànies. L'ampla varietat de requisits no es suportada per les xarxes de quarta generació (4G) actuals, per el qual es va fer necessari un nou paradigma de xarxes sense fil. Amb la promesa de amplies quantitats d'ample de banda, l'estàndard 5G contempla utilitzar freqüències a la banda de mil·limètriques. Esta banda presenta l'inconvenient d'experimentar grans pèrdues de propagació, que s'accentuen en cas de bloqueigs. L'apertura de les bandes de mil·limètriques va atraure l'interès tant de l'industria com de l'acadèmia en plantejar solucions per a donar servei en estes bandes. En els últims anys s'han presentat infinitat de treballs basats en sistemes amb múltiples antenes o MIMO, per a conformar els senyals transmesos o rebuts en feixos apuntant en determinades direccions d'interès. El guany de feix es pot utilitzar per a compensar les pèrdues de propagació, assegurant la viabilitat de les comunicacions en la banda de mil·limètriques. No obstant això, s'ha detectat una preocupant manca d'estudis sobre la viabilitat d'estos sistemes en entorns mòbils i dinàmics, amb obstacles que bloquejen els feixos i facen necessari que el sistema es reconfigure. El present treball de Tesi pretén cobrir este espai buit i des d'un punt de vista pràctic, es proposen mecanismes de gestió dels feixos per a ser el seguiment utilitzant els recursos i mecanismes dels que disposa l'estàndard 5G. D'esta manera, les solucions aportades es basen en la utilització eficient dels reports de mesures dels senyals de referència del enllaç descendent. En primer lloc, esta Tesi recull una anàlisi minuciosa de l'estat de l'art on es corrobora la necessitat de aportar solucions de seguiment de feixos per a comunicacions en la banda de freqüències mil·limètriques. A més a més, s'estudien els diferents mecanismes definits a l'estàndard 5G i que possibiliten el seguiment. Cap destacar que l'estàndard no defineix un mecanisme únic, si no que deixa la porta oberta a presentar propostes. Una vegada conegudes les tecnologies, l'estudi es centra en l'impacte del seguiment sobre les prestacions a nivell de xarxa i d'enllaç. Este estudi es realitza sobre un sistema MIMO punt a punt, en una única estació base i un terminal mòbil desplaçant-se en un entorn urbà. En base a simulacions d'extrem a extrem, es quantifica l'índex de seguiment de feix i com l'anomenat seguiment afecta a la relació senyal a soroll més interferència (SINR) i a la taxa instantània de transmissió de l'usuari. Les solucions de seguiment de feixos propostes a la Tesi es poden classificar en dos categories. A la primera categoria, el seguiment de feixos es realitza en base als reports de mesures dels senyals de referència. Independentment de la velocitat, s'arriba a una taxa de seguiment del 91% amb poca penalització de taxa de transmissió si els feixos d'interès es mesuren amb una periodicitat menor a 20 ms. A la segona categoria pertanyen els algoritmes que utilitzen fonts d'informació externes. Dins d'aquesta categoria es proposa un fingerprinting que relaciona un parell de feixos amb la ubicació de l'usuari, i a banda un model d'intel·ligència artificial (IA) que preveu el feix a utilitzar. El fingerprinting ofereix el mateix rendiment. Però, esta solució es molt sensible a errors i requereix considerar tots els casos possibles, fent-la tecnològicament inviable. En canvi, el / [EN] The arrival of the next generation of mobile communication standards, the so-called Fifth Generation (5G), is already a reality. The first commercial networks have begun to be deployed, and they focus on providing higher data rates. However, the 5G standard goes much further from that and aims at providing support to new services which will revolutionise the society. These new services impose a high level of requirements not only in terms of the data traffic speed, but also in terms of very low latency or incredibly large number of simultaneous connections. This wide variety of requirements cannot be technologically supported by the current Fourth Generation (4G) networks, so it became necessary to move forward with a new paradigm for wireless networks. With the promise of large amounts of bandwidth, in the order of GHz, the 5G standard contemplates the use of frequencies in the commonly known Millimetre Wave (mmWave) band. The mmWave band experiences large propagation losses, which are accentuated in blockage events. Regulatory activities worldwide in the mmWave bands attracted the interest of both the industry and the academia. In the last few years, a tremendous number of contributions on mmWave propagation studies and networks have appeared, most of them based on Multiple-Input Multiple-Output (MIMO) solutions. MIMO architectures allow to beamform, which focuses the radiated energy on certain directions of interest called beams. The additional beam gain compensates the high propagation losses, ensuring the viability of the communications in the mmWave band. There is an evident lack of viability studies of mmWave MIMO systems in mobile and highly-dynamic environments, where obstacles may block beams and forcing frequent re-configurations. This Thesis work aims to fill this gap from a practical approach. This Thesis proposes beam management mechanisms utilising the mechanisms and resources offered by the Third Generation Partnership Project (3GPP) 5G radio access standard: 5G New Radio (NR). The practical solutions are based on the efficient use of measurement reports of standardised downlink Reference Signals (RS). In first place, this Thesis provides a thorough state-of-the-art analysis and corroborates the need of adopting beam tracking solutions for mmWave networks. Then, a complete overview of the 5G standard mechanisms that enable beam tracking is given. The NR standard does not define a standardised mechanism for beam tracking, leaving the door open to proposals to carry out such monitoring. Once the technologies have been identified, the Thesis continues with assessing the impact of the beam tracking strategies on the network and link-level performance. The study is focused on individual point-to-point mmWave links in a realistic urban environment. Based on end-to-end network simulations, the Thesis is interested in assessing the beam tracking success ratio and how beam misalignment affects the perceived Signal to Noise plus Interference Ratio (SINR) and user throughput at pedestrian and vehicular speeds. The beam tracking solutions proposed in this Thesis fall into two categories. The first category monitors beams based on measuring and reporting beamformed RS. Regardless of the speed, this beam tracking category provides up to 91 % tracking performance, with little throughput reduction if the beams of interest are measured with a periodicity below 20 ms. Beam tracking in the second category relies on external information sources. Within this category, this Thesis proposes a fingerprinting database relating beams to the user position and a machine learning (ML) model. Fingerprinting beam tracking is technologically viable and provides similar performance levels. However, this solution is very sensitive to errors and requires considering all possible situations. The ML beam tracking, which makes predictions with a 16 % of estimation error for the reference data set. / I want to thank the Spanish Ministry of Education and Professional Formation for funding this Thesis work with an official pre-doctoral contract grant. / Herranz Claveras, C. (2019). Beam Tracking Strategies for 5G New Radio Networks Operating in the Millimetre Wave Bands [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/130845