Network data are ubiquitous in modern machine learning, with tasks of interest including node classification, node clustering and link prediction being performed on diverse data sets, including protein-protein interaction networks, social networks and citation networks. A frequent approach to approaching these tasks begins by learning an Euclidean embedding of the network, to which machine learning algorithms developed for vector-valued data are applied. For large networks, embeddings are learned using stochastic gradient methods where the sub-sampling scheme can be freely chosen. This distinguishes it from the setting of traditional i.i.d data where there is essentially only one way of subsampling the data - selecting the data points uniformly and without replacement. Despite the strong empirical performance when using embeddings produced in such a manner, they are not well understood theoretically, particularly with regards to the role of the sampling scheme. Here, we develop a unifying framework which encapsulates representation learning methods for networks which are trained via performing gradient updates obtained by subsampling the network, including random-walk based approaches such as node2vec. In particular, we prove, under the assumption that the network has an exchangeable law, that the distribution of the learned embedding vectors asymptotically decouples. We characterize the asymptotic distribution of the learned embedding vectors, and give the corresponding rates of convergence, which depend on factors such as the sampling scheme, the choice of loss function, and the choice of embedding dimension. This provides a theoretical foundation to understand what the embedding vectors represent and how well these methods perform on downstream tasks; in particular, we apply our results to argue that the embedding vectors produced by node2vec can be used to perform weakly consistent community detection.

<div>Federated learning (FL) is a decentralized privacy-preserving learning technique in which clients learn a joint collaborative model through a central aggregator without sharing their data. In this setting, all clients learn a single common predictor (FedAvg), which does not generalize well on each client's local data due to the statistical data heterogeneity among clients. In this paper, we address this problem with PersFL, a discrete two-stage personalized learning algorithm. In the first stage, PersFL finds the optimal teacher model of each client during the FL training phase. In the second stage, PersFL distills the useful knowledge from optimal teachers into each user's local model. The teacher model provides each client with some rich, high-level representation that a client can easily adapt to its local model, which overcomes the statistical heterogeneity present at different clients. We evaluate PersFL on CIFAR-10 and MNIST datasets using three data-splitting strategies to control the diversity between clients' data distributions.</div><div><br></div><div>We empirically show that PersFL outperforms FedAvg and three state-of-the-art personalization methods, pFedMe, Per-FedAvg and FedPer on majority data-splits with minimal communication cost. Further, we study the performance of PersFL on different distillation objectives, how this performance is affected by the equitable notion of fairness among clients, and the number of required communication rounds. We also build an evaluation framework with the following modules: Data Generator, Federated Model Generation, and Evaluation Metrics. We introduce new metrics for the domain of personalized FL, and split these metrics into two perspectives: Performance, and Fairness. We analyze the performance of all the personalized algorithms by applying these metrics to answer the following questions: Which personalization algorithm performs the best in terms of accuracy across all the users?, and Which personalization algorithm is the fairest amongst all of them? Finally, we make the code for this work available at https://tinyurl.com/1hp9ywfa for public use and validation.</div>

Due to privacy and regulatory reasons, sharing data between institutions can be difficult. Because of this, real-world data are not fully exploited by machine learning (ML). An emerging method is to train ML models with federated learning (FL) which enables clients to collaboratively train ML models without sharing raw training data. We explored peer-to-peer FL by extending a prominent centralized FL algorithm called Fedavg to function in a peer-to-peer setting. We named this extended algorithm FedavgP2P. Deep neural networks at 100 simulated clients were trained to recognize digits using FedavgP2P and the MNIST data set. Scenarios with IID and non-IID client data were studied. We compared FedavgP2P to Fedavg with respect to models' convergence behaviors and communication costs. Additionally, we analyzed the connection between local client computation, the number of neighbors each client communicates with, and how that affects performance. We also attempted to improve the FedavgP2P algorithm with heuristics based on client identities and per-class F1-scores. The findings showed that by using FedavgP2P, the mean model convergence behavior was comparable to a model trained with Fedavg. However, this came with a varying degree of variation in the 100 models' convergence behaviors and much greater communications costs (at least 14.9x more communication with FedavgP2P). By increasing the amount of local computation up to a certain level, communication costs could be saved. When the number of neighbors a client communicated with increased, it led to a lower variation of the models' convergence behaviors. The FedavgP2P heuristics did not show improved performance. In conclusion, the overall findings indicate that peer-to-peer FL is a promising approach.

Intrusion Detection Systems are vital for computer networks as they protect against attacks that lead to privacy breaches and data leaks. Over the years, researchers have formulated intrusion detection systems (IDS) using machine learning and/or deep learning to detect network anomalies and identify four main attacks namely, Denial of Service (DoS), Probe, Remote to Local (R2L) and User to Root (U2R). However, the existing models are efficient in detecting just few of the aforementioned attacks while having inadequate detection rates for the rest. This deficiency makes it difficult to choose an appropriate IDS model when a user does not know what attacks to expect. Thus, there is a need for an IDS model that can detect, with uniform efficiency, all the four main classes of network intrusions. This research is aimed at exploring a machine learning approach to an intrusion detection model that can detect DoS, Probe, R2L and U2R attack classes with uniform and high efficiency. A multilayer perceptron was trained in an ensemble with J48 decision tree. The resultant ensemble learning model achieved over 85% detection rates for each of DoS, probe, R2L, and U2R attacks.

<p>Commercial buildings consume more than 19% of the total energy consumption in the United States. Most of this energy is consumed by the HVAC and shading/lighting systems inside these buildings. The main purpose of such systems is to provide satisfactory thermal and visual environments for occupants working inside these buildings. Providing satisfactory thermal/visual conditions in indoor environments is critical since it directly affects occupants’ comfort, health and productivity and has a significant effect on energy performance of the buildings. </p> <p>Therefore, efficiently learning occupants’ preferences is of prime importance to address the dual energy challenge of reducing energy usage and providing occupants with comfortable spaces at the same time. The objective of this thesis is to develop robust and easy to implement algorithms for learning and eliciting thermal and visual preferences of office occupants from limited data. As such, the questions studied in this thesis are: 1) How can we exploit concepts from utility theory to model (in a Bayesian manner) the hidden thermal and visual utility functions of different occupants? Our central hypothesis is that an occupant’s preference relation over different thermal/visual states of the room can be described using a scalar function of these states, which we call the “occupant’s thermal/visual utility function.” 2) By making use of formalisms in Bayesian decision theory, how can we learn the maximally preferred thermal/visual states for different occupants without requiring unnecessary or excessive efforts from occupants and/or the building engineers? The challenge here is to minimize the number of queries posed to the occupants to learn the maximally preferred thermal/visual states for each occupant. 3) Inferring preferences of occupants based on their responses to the thermal/visual comfort-based questionnaire surveys is intrusive and expensive. Contrary to this, how can we learn the thermal/visual preferences of occupants from cheap and non-intrusive human-building interactions’ data? 4) Lastly, based on the observation that the occupant population decompose into different clusters of occupants having similar preferences, how can we exploit the collective information obtained from the similarities in the occupants’ behavior? This thesis presents viable answers to the aforementioned questions in the form of probabilistic graphical models/frameworks. In future, I hope that these frameworks would prove to be an important step towards the development of intelligent thermal/visual systems which would be able to respond to occupants’ personalized comfort needs. Furthermore, in order to encourage the use of these frameworks and ensure reproducibility in results,various implementations of this work (namely GPPref, GPElicit and GPActToPref) are published as open-source Python packages.</p><br>

L'imagerie par résonance magnétique du processus de diffusion (IRMd) de l'eau dans le cerveau a connu un succès fulgurant au cours de la décennie passée pour cartographier les connexions cérébrales. C'est toujours aujourd'hui la seule technique d'investigation de la connectivité anatomique du cerveau humain in vivo. Mais depuis quelques années, il a été démontré que l'IRMd est également un outil unique de biopsie virtuelle in vivo en permettant de sonder la composition du parenchyme cérébral également in vivo. Toutefois, les modèles développés à l'heure actuelle (AxCaliber, ActiveAx, CHARMED) reposent uniquement sur la modélisation des membranes axonales à l'aide de géométries cylindriques, et restent trop simplistes pour rendre compte précisément de l'ultrastructure de la substance blanche et du processus de diffusion dans l’espace extra-axonal. Dans un premier temps, un modèle analytique plus réaliste de la substance blanche cérébrale tenant compte notamment de la dépendance temporelle du processus de diffusion dans le milieu extra-axonal a été développé. Un outil de décodage complexe permettant de résoudre le problème inverse visant à estimer les divers paramètres de la cytoarchitecture de la substance blanche à partir du signal IRMd a été mis en place en choisissant un schéma d'optimisation robuste pour l'estimation des paramètres. Dans un second temps, une approche Big Data a été conduite pour améliorer le décodage de la microstructure cérébrale. Un outil de création de tissus synthétiques réalistes de la matière blanche a été développé, permettant de générer très rapidement un grand nombre de voxels virtuels. Un outil de simulation ultra-rapide du processus de diffusion des particules d'eau dans ces voxels virtuels a ensuite été mis en place, permettant la génération de signaux IRMd synthétiques associés à chaque voxel du dictionnaire. Un dictionnaire de voxels virtuels contenant un grand nombre de configurations géométriques rencontrées dans la matière blanche cérébrale a ainsi été construit, faisant en particulier varier le degré de gonflement de la membrane axonale qui peut survenir comme conséquence de pathologies neurologiques telles que l’accident vasculaire cérébral. L'ensemble des signaux simulés associés aux configurations géométriques des voxels virtuels dont ils sont issus a ensuite été utilisé comme un jeu de données permettant l'entraînement d'un algorithme de machine learning pour décoder la microstructure de la matière blanche cérébrale à partir du signal IRMd et estimer le degré de gonflement axonal. Ce décodeur a montré des résultats de régression encourageants sur des données simulées inconnues, montrant le potentiel de l’approche computationnelle présentée pour cartographier la microstructure de tissus cérébraux sains et pathologiques in vivo. Les outils de simulation développés durant cette thèse permettront, en utilisant un algorithme de recalage difféomorphe de propagateurs de diffusion d’ensemble également développé dans le cadre de cette thèse, de construire un atlas probabiliste des paramètres microstructuraux des faisceaux de matière blanche. / Diffusion Magnetic Resonance Imaging of water in the brain has proven very useful to establish a cartography of brain connections. It is the only in vivo modality to study anatomical connectivity. A few years ago, it has been shown that diffusion MRI is also a unique tool to perform virtual biopsy of cerebral tissues. However, most of current analytical models (AxCaliber, ActiveAx, CHARMED) employed for the estimation of white matter microstructure rely upon a basic modeling of white matter, with axons represented by simple cylinders and extra-axonal diffusion assumed to be Gaussian. First, a more physically plausible analytical model of the human brain white matter accounting for the time-dependence of the diffusion process in the extra-axonal space was developed for Oscillating Gradient Spin Echo (OGSE) sequence signals. A decoding tool enabling to solve the inverse problem of estimating the parameters of the white matter microstructure from the OGSE-weighted diffusion MRI signal was designed using a robust optimization scheme for parameter estimation. Second, a Big Data approach was designed to further improve the brain microstructure decoding. All the simulation tools necessary to construct computational models of brain tissues were developed in the frame of this thesis. An algorithm creating realistic white matter tissue numerical phantoms based on a spherical meshing of cell shapes was designed, enabling to generate a massive amount of virtual voxels in a computationally efficient way thanks to a GPU-based implementation. An ultra-fast simulation tool of the water molecules diffusion process in those virtual voxels was designed, enabling to generate synthetic diffusion MRI signal for each virtual voxel. A dictionary of virtual voxels containing a huge set of geometrical configurations present in white matter was built. This dictionary contained virtual voxels with varying degrees of axonal beading, a swelling of the axonal membrane which occurs after strokes and other pathologies. The set of synthetic signals and associated geometrical configurations of the corresponding voxels was used as a training data set for a machine learning algorithm designed to decode white matter microstructure from the diffusion MRI signal and estimate the degree of axonal beading. This decoder showed encouraging regression results on unknown simulated data, showing the potential of the presented approach to characterize the microstructure of healthy and injured brain tissues in vivo. The microstructure decoding tools developed during this thesis will in particular be used to characterize white matter tissue microstructural parameters (axonal density, mean axonal diameter, glial density, mean glial cells diameter, microvascular density ) in short and long bundles. The simulation tools developed in the frame of this thesis will enable the construction of a probabilistic atlas of the white matter bundles microstructural parameters, using a mean propagator based diffeomorphic registration tool also designed in the frame of this thesis to register each individual.

Indiana University-Purdue University Indianapolis (IUPUI) / In the 21st century, there is an increasing number of vehicles on the road as well as a limited road infrastructure. These aspects culminate in daily challenges for the average commuter due to congestion and slow moving traffic. In the United States alone, it costs an average US driver $1200 every year in the form of fuel and time. Some positive steps, including (a) introduction of the push notification system and (b) deploying more law enforcement troops, have been taken for better traffic management. However, these methods have limitations and require extensive planning. Another method to deal with traffic problems is to track the congested area in a city using social media. Next, law enforcement resources can be re-routed to these areas on a real-time basis. Given the ever-increasing number of smartphone devices, social media can be used as a source of information to track the traffic-related incidents. Social media sites allow users to share their opinions and information. Platforms like Twitter, Facebook, and Instagram are very popular among users. These platforms enable users to share whatever they want in the form of text and images. Facebook users generate millions of posts in a minute. On these platforms, abundant data, including news, trends, events, opinions, product reviews, etc. are generated on a daily basis. Worldwide, organizations are using social media for marketing purposes. This data can also be used to analyze the traffic-related events like congestion, construction work, slow-moving traffic etc. Thus the motivation behind this research is to use social media posts to extract information relevant to traffic, with effective and proactive traffic administration as the primary focus. I propose an intuitive two-step process to utilize Twitter users' posts to obtain for retrieving traffic-related information on a real-time basis. It uses a text classifier to filter out the data that contains only traffic information. This is followed by a Part-Of-Speech (POS) tagger to find the geolocation information. A prototype of the proposed system is implemented using distributed microservices architecture.

We show how physics-informed neural networks can be used to solve compressible \NS equations in protoplanetary disks. While young planets form in protoplanetary disks, because of the limitation of current techniques, direct observations of them are challenging. So instead, existing methods infer the presence and properties of planets from the disk structures created by disk-planet interactions. Hydrodynamic and radiative transfer simulations play essential roles in this process. Currently, the lack of computer resources for these expensive simulations has become one of the field's main bottlenecks. To solve this problem, we explore the possibility of using physics-informed neural networks, a machine learning method that trains neural networks using physical laws, to substitute the simulations. We identify three main bottlenecks that prevent the physics-informed neural networks from achieving this goal, which we overcome by hard-constraining initial conditions, scaling outputs and balancing gradients. With these improvements, we reduce the relative L2 errors of predicted solutions by 97% ~ 99\% compared to the vanilla PINNs on solving compressible NS equations in protoplanetary disks. / Graduate / 2022-12-10

No description available.

Many developed machine learning models are not used in production applications as several challenges must be solved to develop and deploy ML models. Manual reimplementation and heterogeneous environments increase the effort required to develop an ML model or improve an existing one, considerably slowing down the overall process. Furthermore, it is required that a model is constantly monitored to ensure high-quality predictions and avoid possible drifts or biases. MLOps processes solve these challenges and streamline the development and deployment process by covering the whole life cycle of ML models. Even if the research area of MLOps, which applies DevOps principles to ML models, is relatively new, several researchers have already developed abstract MLOps process models. Research for cases with multiple collaboration partners is rare. This research project aims to develop an MLOps process for cases involving multiple collaboration partners. Hence, a case study is conducted with the cooperation of Aimo and LNU as a single case. Aimo requires ML models for their application and collaborates with LNU regarding this demand. LNU develops ML models based on the provided data, which Aimo integrates into their application afterward. This case is analyzed in-depth to identify challenges and the current process. These results are required to elaborate a suitable MLOps process for the case, which also considers the handover of artifacts between the collaboration partners. This process is derived from the already existing general MLOps process models. It is also instantiated to generate a benefit for the case and evaluate the feasibility of the MLOps process. Required components are identified, and existing MLOps tools are collected and compared, leading to the selection of suitable tools for the case. A project template is implemented and applied to an ML model project of the case to show the feasibility. As a result, this research project provides a concrete MLOps process. Besides that, several artifacts were elaborated, such as a project template for ML models in which the selected toolset is applied. These results mainly fit the analyzed case. Nevertheless, several findings are also generalizable such as the identified challenges. The compared alternatives and the generally applied method to elaborate an MLOps process can also be applied to other settings. This is also the case for several artifacts of this project, such as the tool comparison table and the applied process to select suitable tools. This case study shows that it is possible to set up MLOps processes with a high maturity level in situations where multiple cooperation partners are involved and artifacts need to be transferred among them.