The growing adoption of millimeter wave (mmWave) turns efficient beamforming and beam management procedures into key enablers for 5th Generation (5G) and Beyond 5G (B5G) mobile networks. Recent research has sought to optimize beam management in modern Radio Access Network (RAN) architectures, where open, virtualized, disaggregated and multi-vendor environments are considered, and management platforms allow the use of Artificial Intelligence (AI) and Machine Learning (ML)-based solutions. Moreover, beam management represents some fundamental use cases defined by Open RAN Alliance (O-RAN). This work analyses beam management strategies in Open RAN and proposes solutions for codebook-based mmWave systems inspired by two use cases from O-RAN: the Grid of Beams (GoB) Optimization and the AI/ML-assisted Beam Selection. For the GoB Optimization use case, a scenario subject to constraints on the use of the full GoB due to overhead during beam selection is considered. An Advantage Actor Critic (A2C) learning-based framework is proposed to optimize the GoB, as well as the transmission power in a mmWave network. The proposed technique improves Energy Efficiency (EE) and ensures fair coverage is maintained. The simulations show that A2C-based joint optimization of GoB and transmission power is more effective than using Equally Spaced Beams (ESB) and fixed power, or the optimization of GoB and transmission power disjointly. Compared to the ESB and fixed transmission power strategy, the proposed approach achieves more than twice the average EE in the scenarios under test, and it is closer to the maximum theoretical EE. In the case of the AI/ML-assisted Beam Selection use case, the overhead during beam selection is addressed by a multi-modal sensing-aided ML-based method. When using sensing information sources external to the RAN in a multi-vendor disaggregated environment, such methods must account for privacy and data ownership issues. A Distributed Machine Learning (DML) strategy based on Split Learning (SL) is proposed to this end. The solution can cope with deployment challenges in novel RAN architectures and is applied to single and multi-level beam selection decisions, where the latter considers hierarchical codebook structures. With the proposed approach, accuracy levels above 90% can be achieved, while overhead decreases by 85% or more. SL achieves performance comparable to the centralized learning-based strategies, with the added value of accounting for privacy and data ownership issues.

Cooperative multi-agent systems (MAS) are finding applications in a wide variety of domains, including sensor networks, robotics, distributed control, collaborative decision support systems, and data mining. A cooperative MAS consists of a group of autonomous agents that interact with one another in order to optimize a global performance measure. A central challenge in cooperative MAS research is to design distributed coordination policies. Designing optimal distributed coordination policies offline is usually not feasible for large-scale complex multi-agent systems, where 10s to 1000s of agents are involved, there is limited communication bandwidth and communication delay between agents, agents have only limited partial views of the whole system, etc. This infeasibility is either due to a prohibitive cost to build an accurate decision model, or a dynamically evolving environment, or the intractable computation complexity. This thesis develops a multi-agent reinforcement learning paradigm to allow agents to effectively learn and adapt coordination policies in complex cooperative domains without explicitly building the complete decision models. With multi-agent reinforcement learning (MARL), agents explore the environment through trial and error, adapt their behaviors to the dynamics of the uncertain and evolving environment, and improve their performance through experiences. To achieve the scalability of MARL and ensure the global performance, the MARL paradigm developed in this thesis restricts the learning of each agent to using information locally observed or received from local interactions with a limited number of agents (i.e., neighbors) in the system and exploits non-local interaction information to coordinate the learning processes of agents. This thesis develops new MARL algorithms for agents to learn effectively with limited observations in multi-agent settings and introduces a low-overhead supervisory control framework to collect and integrate non-local information into the learning process of agents to coordinate their learning. More specifically, the contributions of already completed aspects of this thesis are as follows: Multi-Agent Learning with Policy Prediction: This thesis introduces the concept of policy prediction and augments the basic gradient-based learning algorithm to achieve two properties: best-response learning and convergence. The convergence property of multi-agent learning with policy prediction is proven for a class of static games under the assumption of full observability. MARL Algorithm with Limited Observability: This thesis develops PGA-APP, a practical multi-agent learning algorithm that extends Q-learning to learn stochastic policies. PGA-APP combines the policy gradient technique with the idea of policy prediction. It allows an agent to learn effectively with limited observability in complex domains in presence of other learning agents. The empirical results demonstrate that PGA-APP outperforms state-of-the-art MARL techniques in both benchmark games. MARL Application in Cloud Computing: This thesis illustrates how MARL can be applied to optimizing online distributed resource allocation in cloud computing. Empirical results show that the MARL approach performs reasonably well, compared to an optimal solution, and better than a centralized myopic allocation approach in some cases. A General Paradigm for Coordinating MARL: This thesis presents a multi-level supervisory control framework to coordinate and guide the agents' learning process. This framework exploits non-local information and introduces a more global view to coordinate the learning process of individual agents without incurring significant overhead and exploding their policy space. Empirical results demonstrate that this coordination significantly improves the speed, quality and likelihood of MARL convergence in large-scale, complex cooperative multi-agent systems. An Agent Interaction Model: This thesis proposes a new general agent interaction model. This interaction model formalizes a type of interactions among agents, called {\em joint-even-driven} interactions, and define a measure for capturing the strength of such interactions. Formal analysis reveals the relationship between interactions between agents and the performance of individual agents and the whole system. Self-Organization for Nearly-Decomposable Hierarchy: This thesis develops a distributed self-organization approach, based on the agent interaction model, that dynamically form a nearly decomposable hierarchy for large-scale multi-agent systems. This self-organization approach is integrated into supervisory control framework to automatically evolving supervisory organizations to better coordinating MARL during the learning process. Empirically results show that dynamically evolving supervisory organizations can perform better than static ones. Automating Coordination for Multi-Agent Learning: We tailor our supervision framework for coordinating MARL in ND-POMDPs. By exploiting structured interaction in ND-POMDPs, this tailored approach distributes the learning of the global joint policy among supervisors and employs DCOP techniques to automatically coordinate distributed learning to ensure the global learning performance. We prove that this approach can learn a globally optimal policy for ND-POMDPs with a property called groupwise observability.

The two key issues of modern Bayesian statistics are: (i) establishing a principled approach for \textit{distilling} a statistical prior distribution that is \textit{consistent} with the given data from an initial believable scientific prior; and (ii) development of a \textit{consolidated} Bayes-frequentist data analysis workflow that is more effective than either of the two separately. In this thesis, we propose generalized empirical Bayes as a new framework for exploring these fundamental questions along with a wide range of applications spanning fields as diverse as clinical trials, metrology, insurance, medicine, and ecology. Our research marks a significant step towards bridging the ``gap'' between Bayesian and frequentist schools of thought that has plagued statisticians for over 250 years. Chapters 1 and 2---based on \cite{mukhopadhyay2018generalized}---introduces the core theory and methods of our proposed generalized empirical Bayes (gEB) framework that solves a long-standing puzzle of modern Bayes, originally posed by Herbert Robbins (1980). One of the main contributions of this research is to introduce and study a new class of nonparametric priors ${\rm DS}(G, m)$ that allows exploratory Bayesian modeling. However, at a practical level, major practical advantages of our proposal are: (i) computational ease (it does not require Markov chain Monte Carlo (MCMC), variational methods, or any other sophisticated computational techniques); (ii) simplicity and interpretability of the underlying theoretical framework which is general enough to include almost all commonly encountered models; and (iii) easy integration with mainframe Bayesian analysis that makes it readily applicable to a wide range of problems. Connections with other Bayesian cultures are also presented in the chapter. Chapter 3 deals with the topic of measurement uncertainty from a new angle by introducing the foundation of nonparametric meta-analysis. We have applied the proposed methodology to real data examples from astronomy, physics, and medical disciplines. Chapter 4 discusses some further extensions and application of our theory to distributed big data modeling and the missing species problem. The dissertation concludes by highlighting two important areas of future work: a full Bayesian implementation workflow and potential applications in cybersecurity. / Statistics

The current period of rapid technological change, particularly in the area of mediated communication, has combined with new philosophies of education and market forces to bring upheaval to the realm of higher education. Technical capabilities exceed our knowledge of whether expenditures on hardware and software lead to corresponding gains in student learning. Educators do not yet possess sophisticated assessments of what we may be gaining or losing as we widen the scope of distributed learning. The purpose of this study was not to draw sweeping conclusions with respect to the costs or benefits of technology in education. The researcher focused on a single issue involved in educational quality: assessing the ability of a student to complete a course. Previous research in this area indicates that attrition rates are often higher in distributed learning environments. Educators and students may benefit from a reliable instrument to identify those students who may encounter difficulty in these learning situations. This study is aligned with research focused on the individual engaged in seeking information, assisted or hindered by the capabilities of the computer information systems that create and provide access to information. Specifically, the study focused on the indicators of completion for students enrolled in video conferencing and Web-based courses. In the final version, the Distributed Learning Survey encompassed thirteen indicators of completion. The results of this study of 396 students indicated that the Distributed Learning Survey represented a reliable and valid instrument for identifying at-risk students in video conferencing and Web-based courses where the student population is similar to the study participants. Educational level, GPA, credit hours taken in the semester, study environment, motivation, computer confidence, and the number of previous distributed learning courses accounted for most of the predictive power in the discriminant function based on student scores from the survey.

Approved for public release; distribution in unlimited. / The thesis research examines the emergence of Sharable Content Object Reference Model (SCORM) architecture currently under development by the Advanced Distributed Learning (ADL) initiative established by the Department of Defense (DoD). SCORM is a collection of specifications adapted from multiple sources to provide a comprehensive suite of E-Learning capabilities that enable interoperability, accessibility, and reusability of Web-based learning content. To understand better the implementation issues of SCORM architecture, the authors analyze all versions of SCORM to understand the evolution of this emerging architecture. It contrasts the evolving requirements for shareable content objects with concerns of copyright issues. The authors address development and implementation issues surrounding the maturation of SCORM architecture and the ADL initiative. The authors recommend that DoD, international, and civilian business partners join in improving E-Learning by embracing technology, such as SCORM, that allows for shareable content objects to be used and reused within civilian and military education and training Learning Management Systems (LMS) across the World Wide Web. / Lieutenant Commander, United States Navy / Lieutenant, United States Naval Reserve

Water monitoring is an essential process that managesthe well-being of freshwater ecosystems. However, it isgenerally an inefficient process as most data collection is donemanually. By combining wireless sensor technology and machinelearning techniques, projects such as iWater aim to modernizecurrent methods. The purpose of the iWater project is to developa network of smart sensors capable of collecting and analyzingwater quality-related data in real time.To contribute to this goal, a comparative study between theperformance of a centralized machine learning algorithm thatis currently used, and a distributed model based on a federatedlearning algorithm was done. The data used for training andtesting both models was collected by a wireless sensor developedby the iWater project. The centralized algorithm was used asthe basis for the developed distributed model. Due to lack ofsensors, the distributed model was simulated by down-samplingand dividing the sensor data into six data sets representing anindividual sensor. The results are similar for both models andthe developed algorithm reaches an accuracy of 98.41 %. / Vattenövervakning är en nödvändig processför att få inblick i sötvattensekosystems välmående. Dessvärreär det en kostsam och tidskrävande process då insamling avdata vanligen görs manuellt. Genom att kombinera trådlössensorteknologi och maskininlärnings algoritmer strävar projektsom iWater mot att modernisera befintliga metoder.Syftet med iWater är att skapa ett nätverk av smarta sensorersom kan samla in och analysera vattenkvalitetsrelaterade datai realtid. För att bidra till projektmålet görs en jämförandestudie mellan den prediktiva noggrannheten hos en centraliseradmaskininlärningsalgoritm, som i nuläget används, och endistribuerad modell baserad på federerat lärande. Data somanvänds för träning och testning av båda modellerna samladesin genom en trådlös sensor utvecklad inom iWater-projektet.Den centraliserade algoritmen användes som grund för denutvecklade distribuerade modellen. På grund av brist på sensorersimulerades den distribuerade modellen genom nedprovtagningoch uppdelning av data i sex datamängder som representerarenskilda sensorer. Resultaten för båda modellerna var liknandeoch den utvecklade algoritmen har en noggrannhet på 98.41 % / Kandidatexjobb i elektroteknik 2021, KTH, Stockholm

The proliferation of web based technologies has resulted in new forms of communities and organizations with enormous implications for design of learning and education. This thesis explores learning occurring within open source software (OSS) communities. OSS communities are a dominant form of organizing in software development with implications not only for innovative product development but also for the training of a large number of software developers. The central catalyst of learning within these communities is expert-novice interactions. These interactions between experts and novices or newcomers are critical for the growth and sustenance of a community and therefore it is imperative that experts are able to provide newcomers requisite advice and support as they traverse the community and develop software. Although prior literature has demonstrated the significance of expert-novice interactions, there are two central issues that have not been examined. First, there is no examination of the role of external events on community interaction, particularly as it relates to experts and novices. Second, the exact nature of expert help, particularly, the quantity of help and whether it helps or hinders newcomer participation has not been studied. This thesis studies these two aspects of expert-novice interaction within OSS communities. The data for this study comes from two OSS communities. The Java newcomer forum was studied as it provided a useful setting for examining external events given the recent changes in Java's ownership. Furthermore, the forum has a rating system which classifies newcomers and experienced members allowing the analysis of expert-novice interactions. The second set of data comes from the MySQL newcomer forum which has also undergone organizational changes and allows for comparison with data from the Java forum. Data were collected by parsing information from the HTML pages and stored in a relational database. To analyze the effect of external events, a natural experiment method was used whereby participation levels were studied around significant events that affected the community. To better understand the changes contextually, an extensive study of major news outlets was also undertaken. Findings from the external event study show significant changes in participation patterns, especially among newcomers in response to key external events. The study also revealed that the changes in participation of newcomers were observed even though other internal characteristics (help giving, expert participation) did not change indicating that external events have a strong bearing on community participation. The effect of expert advice was studied using a logistic regression model to determine how specific participation patterns in discussion threads led to the final response to newcomers. This was supported by social network analysis to visually interpret the participation patterns of experienced members in two different scenarios, one in which the question was answered and the other where it was not. Findings show that higher number of responses from experienced members did not correlate with a response. Therefore, although expert help is essential, non-moderated or unguided help can lead to conflict among experts and inefficient feedback to newcomers. / Master of Science

Approved for public release, distribution is unlimited / This thesis critically analyzes the Sharable Content Object Reference Model (SCORM) within higher education and examines SCORM's limitations within a realistic application environment versus within a theoretical/conceptual platform. The thesis also examines environments better suited for implementation of SCORM technology. In addressing the research questions, it was discovered that from the current standards set forth by Advanced Distributed Learning (ADL), SCORM is not well suited for higher education. SCORM technology will prove of greater utility within the Navy Training environment than in higher education. In their effort to share information, higher education institutions would benefit more from a Content Management System in conjunction with a Learning Management System. Subsequent chapters addressed the limitations of SCORM, provided a comparison of the applicability of SCORM within the separate domains of naval Education and Training, and provided a prototype of a Content Management System for institutions of higher learning. / Lieutenant Commander, Hellenic Navy / Lieutenant, United States Naval Reserve

Stochastic and data-distributed optimization algorithms have received lots of attention from the machine learning community due to the tremendous demand from the large-scale learning and the big-data related optimization. A lot of stochastic and deterministic learning algorithms are proposed recently under various application scenarios. Nevertheless, many of these algorithms are based on heuristics and their optimality in terms of the generalization error is not sufficiently justified. In this talk, I will explain the concept of an optimal learning algorithm, and show that given a time budget and proper hypothesis space, only those achieving the lower bounds of the estimation error and the optimization error are optimal. Guided by this concept, we investigated the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We proposed a novel algorithm named Accelerated Nonsmooth Stochastic Gradient Descent, which exploits the structure of common nonsmooth loss functions to achieve optimal convergence rates for a class of problems including SVMs. It is the first stochastic algorithm that can achieve the optimal O(1/t) rate for minimizing nonsmooth loss functions. The fast rates are confirmed by empirical comparisons with state-of-the-art algorithms including the averaged SGD. The Alternating Direction Method of Multipliers (ADMM) is another flexible method to explore function structures. In the second part we proposed stochastic ADMM that can be applied to a general class of convex and nonsmooth functions, beyond the smooth and separable least squares loss used in lasso. We also demonstrate the rates of convergence for our algorithm under various structural assumptions of the stochastic function: O(1/sqrt{t}) for convex functions and O(log t/t) for strongly convex functions. A novel application named Graph-Guided SVM is proposed to demonstrate the usefulness of our algorithm. We also extend the scalability of stochastic algorithms to nonlinear kernel machines, where the problem is formulated as a constrained dual quadratic optimization. The simplex constraint can be handled by the classic Frank-Wolfe method. The proposed stochastic Frank-Wolfe methods achieve comparable or even better accuracies than state-of-the-art batch and online kernel SVM solvers, and are significantly faster. The last part investigates the problem of data-distributed learning. We formulate it as a consensus-constrained optimization problem and solve it with ADMM. It turns out that the underlying communication topology is a key factor in achieving a balance between a fast learning rate and computation resource consumption. We analyze the linear convergence behavior of consensus ADMM so as to characterize the interplay between the communication topology and the penalty parameters used in ADMM. We observe that given optimal parameters, the complete bipartite and the master-slave graphs exhibit the fastest convergence, followed by bi-regular graphs.

Abstract Resource allocation and interference management in wireless small cell networks have been areas of key research interest in the past few years. Although a large number of research studies have been carried out, the needs for high capacity, reliability, and energy efficiency in the emerging fifth-generation (5G) networks warrants the development of methodologies focusing on ultra-dense and self-organizing small cell network (SCN) scenarios. In this regard, the prime motivation of this thesis is to propose an array of distributed methodologies to solve the problem of joint resource allocation and interference management in SCNs pertaining to different network architectures. The present dissertation proposes and investigates distributed control mechanisms for wireless SCNs mainly in three cases: a backhaul-aware interference management mechanism of the uplink of wireless SCNs, a dynamic cluster-based approach for maximizing the energy efficiency of dense wireless SCNs, and a joint power control and user scheduling mechanism for optimizing energy efficiency in ultra-dense SCNs. Optimizing SCNs, especially in the ultra-dense regime, is extremely challenging due to the severe coupling in interference and the dynamics of both queues and channel states. Moreover, due to the lack of inter-base station/cluster communications, smart distributed learning mechanisms are required to autonomously choose optimal transmission strategies based on local information. To overcome these challenges, an array of distributed algorithms are developed by combining the tools from machine learning, Lyapunov optimization and mean-field theory. For each of the above proposals, extensive sets of simulations have been carried out to validate the performance of the proposed methods compared to conventional models that fail to account for the limitations due to network scale, dynamics of queue and channel states, backhaul heterogeneity and capacity constraints, and the lack of coordination between network elements. The results of the proposed methods yield significant gains of the proposed methods in terms of energy savings, rate improvements, and delay reductions compared to the conventional models studied in the existing literature. / Tiivistelmä Langattomien piensoluverkkojen resurssien allokointi ja häiriön hallinta on ollut viime vuosina tärkeä tutkimuskohde. Tutkimuksia on tehty paljon, mutta uudet viidennen sukupolven (5G) verkot vaativat suurta kapasiteettia, luotettavuutta ja energiatehokkuutta. Sen vuoksi on kehitettävä menetelmiä, jotka keskittyy ultratiheisiin ja itseorganisoituviin piensoluverkkoihin. (SCN). Tämän väitöskirjan tärkein tavoite onkin esittää joukko hajautettuja menetelmiä piensoluverkkojen yhteisten resurssien allokointiin ja häiriön hallintaan, kun käytössä on erilaisia verkkoarkkitehtuureja. Tässä väitöskirjassa ehdotetaan ja tutkitaan hajautettuja menetelmiä langattomien piensoluverkkojen hallintaan kolmessa eri tilanteessa: välityskanavan huomioiva häiriönhallinta menetelmä langattomissa piensoluverkoissa, dynaamisiin klustereihin perustuva malli tiheiden langattomien piensoluverkkojen energiatehokkuuden maksimointiin ja yhteinen tehonsäädön ja käyttäjien allokaatio menetelmä ultratiheiden piensoluverkkojen energiatehokkuuden optimointiin. Ultratiheiden piensoluverkkojen optimointi on erittäin haastavaa häiriön sekä jonojen ja kanavatilojen vahvojen kytkösten vuoksi. Lisäksi, koska klustereilla/tukiasemilla ei ole kommunikaatiota, tarvitaan hajautettuja oppimisalgoritmeja, jotta saadaan itsenäisesti valittua optimaaliset lähetys menetelmät hyödyntäen vain paikallista tietoa. Tämän vuoksi kehitetään useita hajautettuja algoritmeja, jotka hyödyntävät koneoppimista, Lyapunov optimointia ja mean-field teoriaa. Kaikki yllä olevat esitetyt menetelmät on validoitu laajoilla simulaatioilla, joilla on voitu todentaa niiden suorituskyky perinteisiin malleihin verrattuna. Perinteiset mallit eivät pysty ottamaan huomioon verkon laajuuden, jonon ja kanavatilojen dynamiikan, eri välityskanavien ja rajallisen kapasiteetin asettamia rajoituksia sekä verkon elementtien välisen koordinoinnin puuttumista. Esitetyt menetelmät tuottavat huomattavia parannuksia energiansäästöön, siirtonopeuteen ja viiveiden vähentämiseen verrattuna perinteisiin malleihin, joita kirjallisuudessa on tarkasteltu.