When hostile missile raids are launched, protecting allied assets requires that many targets be tracked simultaneously. In these raids, it is possible that the number of missiles could outnumber the sensors available to measure them. In these situations, communication between sensors can be utilized along with dynamic task planning to increase the amount of knowledge available concerning these missiles. Since any allied decisions must depend on the knowledge available from the sensors, it follows that improving the overall knowledge will improve the ability of allies to protect their assets through improved decision making. The goal of the this research effort is to create a Sensor Resource Management (SRM) algorithm to optimize the information available during these missile raids, as well as strengthening the simulation framework required to evaluate the performance of the SRM. The SRM must be capable of near-real-time run time so that it could potentially be deployed in a real-world system. The SRM must be capable of providing time-varying assignments to sensors, allowing more than one target to be observed by a single sensor. The SRM must predict measurements based on sensor models to assess the potential information gain by each assignment. Using these predictions, an optimal allocation of all sensors must be constructed. The initial simulation, upon which this work was built, was capable of simulating a set number of missiles launched simultaneously, providing appropriate charts to display the accuracy of knowledge on each target as well as their predicted impact locations. Communication delays are implemented within the simulation, and sensor models are refined. In refining the sensor models, they are given geometric limitations such as range and viewing angles. Additionally, simulated measurements incorporate geometric considerations to provide more realistic values. The SRM is also improved to account for the details added to the simulation. These improvements include creating assignment schedules and allowing a time-varying numbers of targets. The resulting simulation and SRM are presented, and potential future work is discussed. / text

The demand of mobile users with high data-rate services continues to increase. To satisfy the needs of such mobile users, operators must continue to enhance their existing networks. The radio interface is a well-known bottleneck because the radio spectrum is limited and therefore expensive. Efficient use of the radio spectrum is, therefore, very important. To utilise the spectrum efficiently, any of the channels can be used simultaneously in any of the cells as long as interference generated by the base stations using the same channels is below an acceptable level. In cellular networks based on Orthogonal Frequency Division Multiple Access (OFDMA), inter-cell interference reduces the performance of the link throughput to users close to the cell edge. To improve the performance of cell-edge users, a technique called Coordinated Multi-Point (CoMP) transmission is being researched for use in the next generation of cellular networks. For a network to benefit from CoMP, its utilisation of resources should be scheduled efficiently. The thesis focuses on the resource scheduling algorithm development for CoMP joint transmission scheme in OFDMA-based cellular networks. In addition to the algorithm, the thesis provides an analytical framework for the performance evaluation of the CoMP technique. From the system level simulation results, it has been shown that the proposed resource scheduling based on a joint maximum throughput provides higher spectral efficiency compared with a joint proportional fairness scheduling algorithm under different traffic loads in the network and under different criteria of making cell-edge decision. A hybrid model combining the analytical and simulation approaches has been developed to evaluate the average system throughput. It has been found that the results of the hybrid model are in line with the simulation based results. The benefit of the model is that the throughput of any possible call state in the system can be evaluated. Two empirical path loss models in an indoor-to-outdoor environment of a residential area have been developed based on the measurement data at carrier frequencies 900 MHz and 2 GHz. The models can be used as analytical expressions to estimate the level of interference by a femtocell to a macrocell user in link-level simulations.

Among the many advances and innovations in wireless technologies over the past twenty years, MIMO is perhaps among the most successful. MIMO technology has been evolving over the past two decades. Today, the number of antennas equipped at a base station (BS) or an access point (AP) is increasing, which forms what we call ``many-antenna'' MIMO systems. Many-antenna MIMO will have significant impacts on modern wireless communications, as it will allow numerous wireless applications to operate on the vastly underexplored mid-band and high-band spectrum and is able to deliver ultra-high throughput. Although there are considerable efforts on many-antenna MIMO systems, most of them came from physical (PHY) layer information-theoretic exploitation. There is a lack of investigation of many-antenna MIMO from a networking perspective. On the other hand, new knowledge and understanding begin to emerge at the PHY layer, such as the rank-deficient channel phenomenon. This calls for new theories and models for many-antenna MIMO in a networking environment. In addition, the problem space for many-antenna MIMO systems is much broader and more challenging than conventional MIMO. Reusing existing solutions designed for conventional MIMO systems may suffer from inferior performance or require excessive computation time. The goal of this dissertation is to advance many-antenna MIMO techniques for networking research. We focus on the following two critical areas in the context of many-antenna MIMO networks: (i) DoF-based modeling and (ii) real-time optimization. This dissertation consists of two parts that study these two areas. In the first part, we aim to develop new DoF models and theories under general channel rank conditions for many-antenna MIMO networks, and we explored efficient DoF allocation based on our new DoF model. The main contributions of this part are summarized as follows. New DoF models and theories under general channel rank conditions: Existing DoF-based models in networking community assume that the channel matrix is of full rank. However, this assumption no longer holds when the number of antennas becomes many and the propagation environment is not ideal. In this study, we develop a novel DoF model under general channel rank conditions. In particular, we find that for IC, shared DoF consumption at both transmit and receive nodes is most efficient for DoF allocation, which is contrary to existing unilateral IC models based on full-rank channel assumption. Further, we show that existing DoF models under the full-rank assumption are a special case of our generalized DoF model. The findings of this study pave the way for future research of many-antenna networks under general channel rank conditions. Efficient DoF utilization for MIMO networks: We observes that, in addition to the fact that channel is not full-rank, the strength of signals on different directions in the eigenspace is extremely uneven. This offers us new opportunities to efficiently utilize DoFs in a MIMO network. In this study, we introduce a novel concept called ``effective rank threshold''. Based on this threshold, DoFs are consumed only to cancel strong interferences in the eigenspace while weak interferences are treated as noise in throughput calculation. To better understand the benefits of this approach, we study a fundamental trade-off between network throughput and effective rank threshold for an MU-MIMO network. Our simulation results show that network throughput under optimal rank threshold is significantly higher than that under existing DoF IC models. In the second part, we offered real-time designs and implementations to solve many-antenna MIMO problems for 5G cellular systems. In addition to maximizing a specific optimization objective, we aim at offering a solution that can be implemented in sub-ms to meet requirements in 5G standards. The main contributions of this part are summarized as follows. Turbo-HB---A novel design and implementation for ultra-fast hybrid beamforming: We investigate the beamforming problem under hybrid beamforming (HB) architecture. A major practical challenge for HB is to obtain a solution in 500 $mu$s, which is an extremely stringent but necessary time requirement for its deployment in the field. To address this challenge, we present Turbo-HB---a novel beamforming design under the HB architecture that can obtain the beamforming matrices in about 500 $mu$s. The key ideas of Turbo-HB are two-fold. First, we develop low-complexity SVD by exploiting randomized SVD technique and leveraging channel sparsity at mmWave frequencies. Second, we accelerate the overall computation time through large-scale parallel computation on a commercial off-the-shelf (COTS) GPU platform, with special engineering efforts for matrix operations and minimized memory access. Experimental results show that Turbo-HB is able to obtain the beamforming matrices in 500 $mu$s for an MU-MIMO cellular system while achieving similar or better throughput performance by those state-of-the-art algorithms. mCore+---A sub-millisecond scheduler for 5G MU-MIMO systems: We study a scheduling problem in a 5G NR environment. In 5G NR, an MU-MIMO scheduler needs to allocate RBs and assign MCS for each user at each TTI. In particular, multiple users may be co-scheduled on the same RB under MU-MIMO. In addition, the real-time requirement for determining a scheduling solution is at most 1 ms. In this study, we present a novel scheduler mCore+ that can meet the sub-ms real-time requirement. mCore+ is designed through a multi-phase optimization, leveraging large-scale parallelism. In each phase, mCore+ either decomposes the optimization problem into a large number of independent sub-problems, or reduces the search space into a smaller but more promising subspace, or both. We implement mCore+ on a COTS GPU platform. Experimental results show that mCore+ can obtain a scheduling solution in $sim$500 $mu$s. Moreover, mCore+ can achieve better throughput performance than state-of-the-art algorithms. M3---A sub-millisecond scheduler for multi-cell MIMO networks under C-RAN architecture: We investigate a scheduling problem for a multi-cell environment. Under Cloud Radio Access Network (C-RAN) architecture, the signal processing can be performed cooperatively for multiple cells at a centralized baseband unit (BBU) pool. However, a new resource scheduler is needed to jointly determine RB allocation, MCS assignment, and beamforming matrices for all users under multiple cells. In addition, we aim at finding a scheduling solution within each TTI (i.e., at most 1 ms) to conform to the frame structure defined by 5G NR. To do this, we propose M3---a GPU-based real-time scheduler for a multi-cell MIMO system. M3 is developed through a novel multi-pipeline design that exploits large-scale parallelism. Under this design, one pipeline performs a sequence of operations for cell-edge users to explore joint transmission, and in parallel, the other pipeline is for cell-center users to explore MU-MIMO transmission. For validation, we implement M3 on a COTS GPU. We showed that M3 can find a scheduling solution within 1 ms for all tested cases, while it can significantly increase user throughput by leveraging joint transmission among neighboring cells. / Doctor of Philosophy / MIMO is widely considered to be a major breakthrough in modern wireless communications. MIMO comes in different forms. For conventional MIMO, the number of antennas at a base station (BS) or access point (AP) is typically small (< 8). Today, the number of antennas at a BS/AP is typically ranging from 8 to 64 when the carrier frequency is below 24 GHz. When the carrier frequency is above 24 GHz (e.g., mmWave), the number of antennas can be even larger (> 64). We call today's MIMO systems (typically with $ge$ 8 antennas at some nodes) as ``many-antenna'' MIMO systems, and this will be the focus of this dissertation. Although there exists a considerable amount of works on many-antenna MIMO techniques, most efforts focus on physical (PHY) layer for information-theoretic exploitation. There is a lack of investigation on how to efficiently and effectively utilize many-antenna MIMO from a networking perspective. The goal of this dissertation is to advance many-antenna MIMO techniques for networking research. We focus on the following two critical areas in the context of many-antenna MIMO networks: (i) degree-of-freedom (DoF)--based modeling and (ii) real-time optimization. In the first part, we investigate a novel DoF model under general channel rank conditions for many-antenna MIMO networks. The main contributions of this part are summarized as follows. New DoF models and theories under general channel rank conditions: In this study, we develop a novel DoF model under general channel rank conditions. We show that existing works claiming that unilateral DoF consumption is optimal no longer hold when channel rank is deficient (not full-rank). We find that for IC, shared DoF consumption at both Tx and Rx nodes is the most efficient scheme for DoF allocation. Efficient DoF utilization for MIMO networks: In this study, we proposed a new approach to efficiently utilize DoFs in a MIMO network. The DoFs used to cancel interference are conserved by exploiting the interference signal strength in the eigenspace. Our simulation results show that network throughput under our approach is significantly higher than that under existing DoF IC models. In the second part, we offer real-time designs and implementations to solve many-antenna MIMO problems for 5G cellular systems. The timing performance of these designs is tested in actual wall-clock time. A novel design and implementation for ultra-fast hybrid beamforming: We investigate a beamforming problem under the hybrid beamforming (HB) architecture. We propose Turbo-HB---a novel beamforming design under the HB architecture that can obtain the beamforming matrices in about 500 $mu$s. At the same time, Turbo-HB can achieve similar or better throughput performance by those state-of-the-art algorithms. A sub-millisecond scheduler for 5G multi-user (MU)-MIMO systems: We study a resource scheduling problem in 5G NR. We present a novel scheduler called mCore+ that can schedule time-frequency resources to MU-MIMO users and meet the 500 $mu$s real-time requirement in 5G NR. A sub-millisecond scheduler for multi-cell MIMO networks under C-RAN architecture: We investigate the scheduling problem for a multi-cell environment under a centralized architecture. We present M3---a GPU-based real-time scheduler that jointly determines a scheduling solution among multiple cells. M3 can find the scheduling solution within 1 ms.

[pt] A demanda progressiva por aplicações multimídia distribuídas, caracterizadas por fortes exigências sobre os recursos computacionais, torna evidente a necessidade de provisão de qualidade de serviço (QoS) em cada um dos subsistemas envolvidos, como redes de comunicação e sistemas operacionais. Ao mesmo tempo, tais subsistemas devem ser flexíveis para que possam oferecer novos serviços a aplicações futuras, ou seja, devem ser adaptáveis em tempo de execução. Especificamente, sistemas operacionais de uso geral provêm pouco ou nenhum suporte a QoS e à adaptabilidade dos serviços, impulsionando vários estudos isolados nessas áreas. Observando-se algumas tecnologias implementadas em sistemas operacionais específicos, nota-se que os mecanismos de provisão possuem certas semelhanças funcionais. Assim, este trabalho propõe uma arquitetura adaptável para a provisão de QoS nos subsistemas de rede e de escalonamento de processos de sistemas operacionais, independente de implementação, através da descrição de frameworks genéricos. É demonstrado, também, como os pontos de flexibilização desses frameworks podem ser especializados para a implementação de alguns modelos de QoS. Por último, é proposto um cenário de uso da arquitetura, no qual um sistema operacional de uso geral ligeiramente modificado é utilizado como infra- estrutura para a instanciação dos frameworks de QoS. / [en] The progressive demand for distributed multimedia applications, which are characterized by strong requirements over computational resources, makes evident the need for quality of service (QoS) provisioning in each one of the involved subsystems (e.g. communication networks and operating systems). At the same time, these subsystems must be flexible enough that they can offer new services to future applications, or in other words, they must be adaptable at runtime. Specifically, general-purpose operating systems provide few or no QoS/service adaptability support, what have motivated many isolated studies about these topics. Observing some implemented technologies on specific operating systems,it is noted that the provisioning mechanisms have certain functional similarities.In this way, this work proposes an adaptable architecture for QoS provisioning on networking and process scheduling subsystems of operating systems, through the description of generic frameworks. It is demonstrated how the framework hot-spots can be specialized in order to implement some QoS models. Finally, it is proposed a scenario of use of the architecture, where a bit modified generalpurpose operating system is used as infrastructure for an instantiation of the QoS frameworks.

One critical step to successfully integrate wireless data networks to the high-speed wired backbone is the design of network control policies that efficiently utilize resources to provide Quality of Service (QoS) to the users in the integrated networks. Such a design has remained a challenge since wireless networks are time-varying in nature, not only in terms of user/packet arrivals but also in terms of physical channel conditions and access opportunities. In this thesis, we study the stochastic control of time-varying networks to design efficient scheduling and resource allocation policies. In particular, in Chapter 3, we focus on a broad class of control policies that work based on a pick-and-compare principle for networks with time-varying channels. By trading the throughput for complexity and memory requirement, these policies require less complexity compared to the well-investigated throughput-optimal Generalized Maximum Weight Matching (GMWM) policy and also require only linear-memory storage with the number of data-flows. Through Lyapunov analysis tools, we characterize the stability region and delay performance of the studied policies and show how they vary in response to the channel variations. In Chapter 4, we go into further detail and consider the problem of network control from a new perspective through which we carefully incorporate the time-efficiency of underlying scheduling algorithms. Specifically, we develop a policy that dynamically adjusts the time given to the available scheduling algorithms according to queue-backlog and channel correlations. We study the resulting stability region of developed policy and show that the region is at least as large as the one for any static policy. Finally, motivated by the current under-utilization of wireless spectrum, in Chapter 5, we investigate the control of cognitive radio networks as a special example of networks that provide time-varying access opportunities. We assume that users dynamically join and leave the network and may have different utility functions, or could collaborate for a common purpose. We develop a policy that performs joint admission and resource control and works for any user load, either inside or outside the capacity region. Through Lyapunov Optimization techniques, we show that the developed policy can achieve a utility performance arbitrarily close to the optimality with a tradeoff in the average service delay of admitted users.

One critical step to successfully integrate wireless data networks to the high-speed wired backbone is the design of network control policies that efficiently utilize resources to provide Quality of Service (QoS) to the users in the integrated networks. Such a design has remained a challenge since wireless networks are time-varying in nature, not only in terms of user/packet arrivals but also in terms of physical channel conditions and access opportunities. In this thesis, we study the stochastic control of time-varying networks to design efficient scheduling and resource allocation policies. In particular, in Chapter 3, we focus on a broad class of control policies that work based on a pick-and-compare principle for networks with time-varying channels. By trading the throughput for complexity and memory requirement, these policies require less complexity compared to the well-investigated throughput-optimal Generalized Maximum Weight Matching (GMWM) policy and also require only linear-memory storage with the number of data-flows. Through Lyapunov analysis tools, we characterize the stability region and delay performance of the studied policies and show how they vary in response to the channel variations. In Chapter 4, we go into further detail and consider the problem of network control from a new perspective through which we carefully incorporate the time-efficiency of underlying scheduling algorithms. Specifically, we develop a policy that dynamically adjusts the time given to the available scheduling algorithms according to queue-backlog and channel correlations. We study the resulting stability region of developed policy and show that the region is at least as large as the one for any static policy. Finally, motivated by the current under-utilization of wireless spectrum, in Chapter 5, we investigate the control of cognitive radio networks as a special example of networks that provide time-varying access opportunities. We assume that users dynamically join and leave the network and may have different utility functions, or could collaborate for a common purpose. We develop a policy that performs joint admission and resource control and works for any user load, either inside or outside the capacity region. Through Lyapunov Optimization techniques, we show that the developed policy can achieve a utility performance arbitrarily close to the optimality with a tradeoff in the average service delay of admitted users.

Abstract This thesis concentrates on evaluating and improving the throughput performances of mobile users in high speed vehicles. In particular, high speed train (HST) scenarios are considered. Emphasis is placed on practical designs and methods that take into account distinctive HST characteristics. A two-hop communication link, i.e., base station (BS)-to-HST and HST-to-onboard users (OBUs) is adopted. The main target is to improve the throughput performance on the BS-to-HST communication link, which is assumed to be the main bottleneck in the whole communication link, since the HST-to-OBU communication link is assumed to have good channel quality due to the short link distance with relatively stationary OBUs. The algorithms developed are assessed through link and system level simulations. A theoretical and practical study of the throughput maximization problem in a single and multi-cell multiple-input multiple-output orthogonal frequency-division multiplexing (MIMO-OFDM) train scenario are considered with and without cooperation between train carriages. Two low-complexity transmission schemes based on simple antenna selection (AS) methods with spatial multiplexing (SM) are proposed. The simulation results demonstrate that large antenna arrays with AS and SM transmission strategies have the potential to significantly improve the throughput of the BS-to-train link in HST scenarios. Resource sharing methodologies between the moving relay nodes (MRNs) on the HST and ground macro users (GMUs) were also studied in a multi-cell MIMO-OFDM train scenario. Direct application of existing resource scheduling methods will not be appropriate to efficiently and fairly share resources, since the MRNs and the GMUs have different processing capabilities. Hence, two hybrid resource scheduling methods are analyzed in conjunction with joint and disjoint resource management. The tradeoff between the number of MRNs and receive antennas that should be installed on an HST was also examined in the context of throughput performance and capital expenditure. Results show that joint scheduling does not provide the best overall performance and there is a need to schedule each group of mobile terminals (MTs) separately. Finally, the feasibility of the use of higher frequency bands (HFBs) was examined in HST scenarios. A timer-based beam selection scheme for HST, which does not require any training time to select the appropriate beam is also proposed. The proposed beam selection scheme (PBSS) displays a close performance to the ideal singular value decomposition (SVD) scheme. / Tiivistelmä Tämä väitöskirja keskittyy mobiilikäyttäjien tiedonsiirtonopeuksien arviointiin ja parantamiseen nopeasti liikkuvissa kulkuneuvoissa. Työ käsittelee erityisesti tiedonsiirtoa suurnopeusjunissa. Työssä korostetaan käytännön menetelmiä, jotka ottavat huomioon nopeasti liikkuvien junien tiedonsiirron erityispiirteet. Työssä käytetään kahden hypyn linkkimallia, jossa tiedonsiirto kulkee tukiasemalta junaan ja junasta käyttäjälle, joka on junassa. Päätavoite on parantaa datanopeuksia tukiaseman ja junan välisessä tiedonsiirtolinkissä, jonka uskotaan olevan suurin pullonkaula koko tiedonsiirtolinkissä, koska junan ja lähes paikallaan olevan käyttäjän välinen kanava voidaan olettaa hyvälaatuiseksi linkin lyhyyden vuoksi. Kehitettyjen algoritmien suorituskykyä arvioidaan linkki- ja järjestelmätason simulaatioilla. Työssä tutkitaan tiedonsiirtonopeuden maksimointiongelmaa teoreettisella ja käytännön tasolla yhden ja usean solun MIMO OFDM junaskenaarioissa, joissa junan vaunut tekevät tai eivät tee yhteistyötä. Työssä esitetään kaksi alhaisen kompleksisuuden lähetysmenetelmää, jotka hyödyntävät yksinkertaista antennin valintamenetelmää ja tilatason multipleksointia. Simulointitulokset osoittavat, että suuret antenniryhmät, jotka hyödyntävät näitä lähetysmenetelmiä, voivat parantaa merkittävästi tiedonsiirtonopeutta tukiasemalta junaan päin. Työssä tutkitaan myös resurssien jakomenetelmiä liikkuvien junassa olevien releiden ja maatason makrokäyttäjien välillä monen solun MIMO-OFDM junaskenaariossa. Nykyisten resurssinhallintamenetelmien käyttö ei ole suoraan mahdollista tehokasta ja oikeudenmukaista resurssien jakoa, koska releillä ja makrokäyttäjillä on erilaiset prosessointikyvyt. Tämän vuoksi työssä analysoidaan kahta hybridimenetelmään resurssien skeduloinnille. Tutkimukset selventävät tasapainoa releiden lukumäärän ja junaan asennettavien vastaanotinantennien välillä tiedonsiirtonopeuden ja kustannusten osalta. Tulokset osoittavat, että yhteinen resurssien jako ei saavuta parasta suorituskykyä, eikä ole tarvetta ajoittaa jokaista matkaviestinterminaaliryhmää erikseen. Lopuksi työssä tutkitaan korkeampien taajuusalueiden soveltuvuutta tiedonsiirtoon suurnopeusjunissa. Työssä ehdotetaan ajastinpohjaista keilanvalintamenetelmää, joka ei vaadi opetusjaksoa sopivan keilan valintaan. Ehdotetun menetelmän saavuttama suorituskyky on lähellä ideaalisen singulaariarvohajotelmaa hyödyntävän menetelmän suorituskykyä.

Reinforcement Learning (RL) has in recent years become a core method for sequential decision making in complex dynamical systems, being of great interest to support improvements in scheduling problems. This could prove important to areas in the newer generation of cellular networks. One such area is the base stations scheduler which allocates radio resources to users. This is posed as large-scale optmization problem which needs to be solved in millisecond intervals, while at the same time accounting for multiple, sometimes conflicting, objectives like latency or Quality of Service requirements. In this thesis, multi-objective RL (MORL) solutions are proposed and evaluated in order to identify desired features for novel applications to the scheduling problem. The posed solution classes were tested in common MORL benchmark environments such as Deep Sea Treasure for efficient and informative evaluation of features. It was ultimately tested in environments to solve combinatorial optmization and scheduling problems. The results indicate that outer-loop multi-policy solutions are able to produce models that comply with desired features for scheduling. A multi-policy multi-objective deep Q-network was implemented and showed it can produce an adaptive-at-run-time discrete model, based on an outer-loop approach that calls a single-policy algorithm. The presented approach does not increase in complexity when adding objectives but generally requires larger sampling quantities for convergence. Differing scalarization techniques of the reward was tested, indicating effect on variance that could effect performance in certain environment characteristics. / Försärkningsinlärning som en gångbar metod för sekventiellt beslutsfattande i komplexa dynamiska system har ökat under de senaste åren tack vare förbättrade hårdvaru möjligheter. Intressenter av denna teknik finns bland annat inom telekom-indistrin vars aktörer har som mål att uteveckla nya generationens mobilnätverk. En av de grundläggande funktionerna i en basstation är scheduleraren vars uppgift är att allokera radio resurser till användare i nätverket. Detta ställs med fördel upp som ett optimeringsproblem som nödvändiggör att problemet måste lösas på millisekund nivå samtidigt som den kan ta flera typer av mål i beaktning, såsom QoS krav och latens. I detta examensarbete så presenteras och utvärderas förstärningsinlärnings algoritmer för flera mål inom flera lösningsklasser i syfte att identifiera önskvärda funktioner för nya tillämpningar inom radio resurs schemaläggning. De presenterade lösningsklasserna av algoritmer testades i vanligt förekommande riktmärkesmiljöer för denna typ av teknik såsom Deep Sea Treasure för att på effektivt sätt utvärdera de kvalitéer och funktioner varje algoritm har. Slutligen testades lösningen i miljöer inom kombinatorisk optimering och schemaläggning. Resultaten indikerar att fler-policy lösningar har kapaciteten att producera modeller som ligger inom de krav problemet kräver. Fler-policy modeller baserade på djupa Q-närverk av flera mål kunde framställa adaptiva, diskreta realtidsmodeller. Denna lösning ökar inte komplexiteten när fler mål läggs till men har generellt behov av större mängder samplade preferenser för att konvergera. Olika skaläriseringstekniker av belöningen testades och indikerade att dessa påverkade variansen, vilket i vissa typer av miljö konfigurationer påverkade resultaten.