The last decade has witnessed a tremendous increase in the amount of available biological data. Different technologies for measuring the genome, epigenome, transcriptome, proteome, metabolome, and microbiome in different organisms are producing large amounts of high-dimensional data every day. High-dimensional data provides unprecedented challenges and opportunities to gain a better understanding of biological systems. Unlike other data types, biological data imposes more constraints on researchers. Biologists are not only interested in accurate predictive models that capture complex input-output relationships, but they also seek a deep understanding of these models. In the last few years, deep models have achieved better performance in computational prediction tasks compared to other approaches. Deep models have been extensively used in processing natural data, such as images, text, and recently sound. However, application of deep models in biology is limited. Here, I propose to use deep models for output prediction, dimension reduction, and feature selection of biological data to get better interpretation and understanding of biological systems. I demonstrate the applicability of deep models in a domain that has a high and direct impact on health care. In this research, novel deep learning models have been introduced to solve pressing biological problems. The research shows that deep models can be used to automatically extract features from raw inputs without the need to manually craft features. Deep models are used to reduce the dimensionality of the input space, which resulted in faster training. Deep models are shown to have better performance and less variant output when compared to other shallow models even when an ensemble of shallow models is used. Deep models are shown to be able to process non-classical inputs such as sequences. Deep models are shown to be able to naturally process input sequences to automatically extract useful features. / Ph. D. / The world is generating more data than any time before. The abundance of data provides a great challenge and opportunity to get a better understanding of complex biological systems. The complexity of biological systems mandates better computational models that can make use the different types and formats of biological data. In the last few years, deep models have achieved better performance in computational prediction tasks compared to other approaches. Deep models have been extensively used in processing natural data, such as images, text, and recently sound. In this research, I show that deep learning can be applied to solve different biological problems that are directly related to human health. In this research, deep learning is used to predict which genes are essential for cancer cell survival. Deep learning is used to predict which drug combinations can work together to better treat cancer. Deep learning is used to predict whether two proteins are interacting with each other. This can be helpful for example in finding potential targets of viral proteins inside the human body.

Linux containers are gaining increasing traction in both individual and industrial use. As these containers get integrated into mission-critical systems, real-time detection of malicious cyber attacks becomes a critical operational requirement. However, a little research has been conducted in this area. This research introduces an anomaly-based intrusion detection and remediation system for container-based clouds. The introduced system monitors system calls between the container and the host server to passively detect malfeasance against applications running in cloud containers. We started by applying a basic memory-based machine learning technique to model the container behavior. The same technique was also extended to learn the behavior of a distributed application running in a number of cloud-based containers. In addition to monitoring the behavior of each container independently, the system used prior knowledge for a more informed detection system. We then studied the feasibility and effectiveness of applying a more sophisticated deep learning technique to the same problem. We used a recurrent neural network to model the container behavior. We evaluated the system using a typical web application hosted in two containers, one for the front-end web server, and one for the back-end database server. The system has shown promising results for both of the machine learning techniques used. Finally, we describe a number of incident handling and remediation techniques to be applied upon attack detection. / Ph. D. / Cloud computing plays an important role in our daily lives today. Most of the online services and applications we use are hosted in a cloud environment. Examples include email, cloud storage, online booking systems, and many websites. Typically, a cloud environment would host many of those applications on a single host to maximize efficiency and minimize overhead. To achieve that, cloud service providers, such as Amazon Web Services and Google Cloud Platform, rely on virtual encapsulation environments, such as virtual machines and containers, to encapsulate and isolate applications from other applications running in the cloud. One major concern usually raised when discussing cloud applications is the security of the application and the privacy of the data it handles, e.g. the files stored by the end users on their cloud storage. In addition to firewalls and traditional security measures that attempt to prevent an attack from affecting the application, intrusion detection systems (IDS) are usually used to detect when an application is affected by a successful attack that managed to escape the firewall. Many intrusion detection systems have been introduced to cloud applications using virtual machines, but almost none has been introduced to applications running in containers. In this dissertation, we introduce an intrusion detection system to be deployed by cloud service providers to container-based cloud environments. The system uses machine learning techniques to learn the behavior of the application running in the container and detect when the behavior changes as an indication for a potential attack. Upon detection of the attack, the system applies one of three defense mechanisms to restore the running application to a safe state.

As research in Artificial Intelligence (AI) advances, it is crucial to focus on having seamless communication between humans and machines in order to effectively accomplish tasks. Smooth human-machine communication requires the machine to be sensible and human-like while interacting with humans, while simultaneously being capable of extracting the maximum information it needs to accomplish the desired task. Since a lot of the tasks required to be solved by machines today involve the understanding of images, training machines to have human-like and effective image-grounded conversations with humans is one important step towards achieving this goal. Although we now have agents that can answer questions asked for images, they are prone to failure from confusing input, and cannot ask clarification questions, in turn, to extract the desired information from humans. Hence, as a first step, we direct our efforts towards making Visual Question Answering agents human-like by making them resilient to confusing inputs that otherwise do not confuse humans. Not only is it crucial for a machine to answer questions reasonably, it should also know how to ask questions sequentially to extract the desired information it needs from a human. Hence, we introduce a novel game called the Visual 20 Questions Game, where a machine tries to figure out a secret image a human has picked by having a natural language conversation with the human. Using deep learning techniques like recurrent neural networks and sequence-to-sequence learning, we demonstrate scalable and reasonable performances on both the tasks. / Master of Science / Research in Artificial Intelligence has reached to a point where computers can answer natural freeform questions asked to arbitrary images in a somewhat reasonable manner. These machines are called Visual Question Answering agents. However, they are prone to failure from even a slightly confusing input. For example, for an obviously irrelevant question asked to an image, they would answer something non-sensical instead of recognizing that the question is irrelevant. Furthermore, they also cannot ask questions in turn to humans for clarification or for more information. These shortcomings not only harm their efficacy, but also harm their perceived trust from human users. In order to remedy these problems, we first direct our efforts towards making Visual Question Answering agents capable of identifying an irrelevant question for an image. Next, we also try to train machines to be able to ask questions to extract more information from humans to make an informed decision. We do this by introducing a novel game called the Visual 20 Questions game, where a machine tries to figure out a secret image a human has picked by having a natural language conversation with the human. Deep learning techniques such as sequence-to-sequence learning using recurrent neural networks make it possible for machines to learn how to converse based on a series of conversational exchanges made between two humans. Techniques like reinforcement learning make it possible for machines to better themselves based on rewards it gets for accomplishing a task in a certain way. Using such algorithms, we demonstrate promise towards scalable and reasonable performances on both the tasks.

Despite its potential, 66% of the world's population lacks access to magnetic resonance imaging (MRI). The main factors contributing to the uneven distribution of this imaging modality worldwide are the elevated cost and intricate nature of MRI systems coupled with the high level of knowledge and expertise required for its operation and maintenance. To improve its worldwide accessibility, MRI technology and techniques must undergo modifications to deliver a more cost-effective system that is easier to site and use without compromising on the diagnostic quality of the images. This thesis presents two deep learning methods, ArtifactID and GDCNet, developed for artifact detection and correction and tailored for their integration into accessible MRI systems. ArtifactID is targeted to resource-constrained settings where skilled personnel are scarce. It automates part of the quality assessment step, critical during image acquisition to ensure data quality and the success of downstream analysis or interpretation. This study utilized two types of T1-weighted neuroimaging datasets: publicly available and prospective. Combining the two, ArtifactID successfully identified wrap-around and rigid head motion in multi-field strength and multi-vendor data. We leveraged the public datasets for artifact simulation, model training, and testing. In contrast, prospective datasets were reserved for validation and testing and to assess the models’ performance in data representative of clinical and deployment settings. We trained individual convolutional neural networks for each artifact. The wrap-around models perform binary classification, while the multi-class motion classification model allows distinction between moderate and severe motion artifacts. Our models demonstrated strong agreement with ground truth labels and motion metrics and proved potential for generalization to various data distributions. Furthermore, Grad-CAM heatmaps allowed early identification of failure modes, artifact localization within the image, and fine-tuning the pre-processing steps. GDCNet correction applies to imaging techniques highly susceptible to local B0 deviations and systems whose design entails high B0 inhomogeneity. The method estimates a geometric distortion map by non-linear registration to a reference image. The self-supervised model, consisting of a U-Net and a spatial transform function unit, learned the correction by optimizing the similarity between the distorted and the reference images. We initially developed the tool for distortion correction of echo-planar imaging functional MRI images at 3 T. This method allows dynamic correction of the functional data as a distortion map is estimated for each temporal frame. For this model, we leveraged T1-weighted anatomical images as target images. We trained the model on publicly available datasets and tested it on in-distribution and out-of-distribution datasets consisting of other public datasets unseen during training and a prospectively acquired dataset. Comparing GDCNet to state-of-the-art EPI geometric distortion methods, our technique demonstrated statistically significant improvements in normalized mutual information between the corrected and reference images and 14 times faster processing times without requiring the acquisition of additional sequences for field map estimation. We adapted the GDCNet method for distortion correction of low-bandwidth images acquired in a 47 mT permanent magnet system. These systems are characterized by large B0 spatial inhomogeneity and low signal sensitivity. In this case, the model used high-field images or images acquired with higher acquisition bandwidths as reference. The goal was to exploit the signal-to-noise ratio improvements that low bandwidth acquisition offers while limiting geometric distortion artifacts in the images. We investigated two versions of the model using different similarity loss functions. Both models were trained and tested on an in vitro dataset of image-quality phantoms. Additionally, we evaluated the models’ generalization ability to an in vivo dataset. The models successfully reduced distortions to levels comparable to those of the high bandwidth images in vitro and improved geometric accuracy in vivo. Furthermore, the method indicated robust performance on reference images with large levels of noise. Incorporating the methods presented in this thesis into the software of a clinical MRI system will alleviate some of the barriers currently restricting the democratization of MR technology. First, automating the time-consuming process of artifact identification during image quality assessment will improve scan efficiency and augment expertise on-site by assisting non-skilled personnel. Second, efficient off-resonance correction during image reconstruction will ease the tight B0 homogeneity requirements of magnet design, allowing more compact and lightweight systems that are easier to refrigerate and site.

Diffuse optical tomography (DOT) is an imaging technique that utilizes near-infrared (NIR) light to probe biological tissue and ultimately recover the optical parameters of the tissue. Broadly, the process for image reconstruction in DOT involves three parts: (1) the detected measurements, (2) the modeling of the medium being imaged, and (3) the algorithm that incorporates (1) and (2) to finally estimate the optical properties of the medium. These processes have long been established in the DOT field but are also known to suffer drawbacks. The measurements themselves tend to be susceptible to experimental noise that could degrade reconstructed image quality. Furthermore, depending on the DOT configuration being utilized, the total number of measurements per capture can get very large and add additional computational burden to the reconstruction algorithms. DOT algorithms are reliant on accurate modeling of the medium, which includes solving a light propagation model and/or generating a so-called sensitivity matrix. This process tends to be complex and computationally intensive and, furthermore, does not take into account real system characteristics and fluctuations. Similarly, the inverse algorithms typically utilized in DOT also often take on a high computational volume and complexity, leading to long reconstruction times, and have limited accuracy depending on the measurements, forward model, and experimental system. The purpose of this dissertation is to address and develop computational methods, especially incorporating deep learning, to improve each of these components. First, I evaluated several time-domain data features involving the Mellin and Laplace transforms to incorporate measurements that were robust to noise and sensitive at depth for reconstruction. Furthermore, I developed a method to find the optimal values to use for different imaging depths and scenarios. Second, I developed a neural network that can directly learn the forward problem and sensitivity matrix for simulated and experimental measurements, which allows the computational forward model to adapt to the system's characteristics. Finally, I employed learning-based approaches based on the previous results to solve the inverse problem to recover the optical parameters in a high-speed manner. Each of these components were validated and tested with numerical simulations, phantom experiments, and a variety of in vivo data. Altogether, the results presented in this dissertation depict how these computational approaches lead to an improvement in DOT reconstruction quality, speed, and versatility. It is the ultimate hope that these methods, algorithms, and frameworks developed as a part of this dissertation can be directly used on future data to further validate the research presented here and to further validate DOT as a valuable imaging tool across many applications.

Neuroscience as a field is collecting more data than at any other time in history. The scale of this data allows us to ask fundamental questions about the mechanisms of brain function, the basis of behavior, and the development of disorders. Our ambitious goals as well as the abundance of data being recorded call for reproducible, reliable, and accessible systems to push the field forward. While we have made great strides in building reproducible and accessible machine learning (ML) systems for neuroscience, reliability remains a major issue. In this dissertation, we show that we can leverage existing data and domain expert knowledge to build more reliable ML systems to study animal behavior. First, we consider animal pose estimation, a crucial component in many scientific investigations. Typical transfer learning ML methods for behavioral tracking treat each video frame and object to be tracked independently. We improve on this by leveraging the rich spatial and temporal structures pervasive in behavioral videos. Our resulting weakly supervised models achieve significantly more robust tracking. Our tools allow us to achieve improved results when we have imperfect, limited data while requiring users to label fewer training frames and speeding up training. We can more accurately process raw video data and learn interpretable units of behavior. In turn, these improvements enhance performance on downstream applications. Next, we consider a ubiquitous approach to (attempt to) improve the reliability of ML methods, namely combining the predictions of multiple models, also known as deep ensembling. Ensembles of classical ML predictors, such as random forests, improve metrics such as accuracy by well-understood mechanisms such as improving diversity. However, in the case of deep ensembles, there is an open methodological question as to whether, given the choice between a deep ensemble and a single neural network with similar accuracy, one model is truly preferable over the other. Via careful experiments across a range of benchmark datasets and deep learning models, we demonstrate limitations to the purported benefits of deep ensembles. Our results challenge common assumptions regarding the effectiveness of deep ensembles and the “diversity” principles underpinning their success, especially with regards to important metrics for reliability, such as out-of-distribution (OOD) performance and effective robustness. We conduct additional studies of the effects of using deep ensembles when certain groups in the dataset are underrepresented (so-called “long tail” data), a setting whose importance in neuroscience applications is revealed by our aforementioned work. Altogether, our results demonstrate the essential importance of both holistic systems work and fundamental methodological work to understand the best ways to apply the benefits of modern machine learning to the unique challenges of neuroscience data analysis pipelines. To conclude the dissertation, we outline challenges and opportunities in building next-generation ML systems.

Data serves as the foundation in building effective deep learning algorithms, yet the process of annotation and curation to maintain high data quality is time-intensive. The challenges arise from the vast diversity and large amount of data, and the inherent complexity in labeling each sample. Then, relying on manual effort to construct high-quality data is implausible and not sustainable in the real world. Instead, this thesis introduces a set of novel techniques to effectively learn from the data with less curation, which is more practical in building AI applications. In this thesis, we systematically study different directions in learning from low-quality data, with a specific focus on visual understanding and being robust to complicated label bias & noise. We first examine the bias exhibited in the whole dataset for image classification, and derive the debiasing algorithms based on representation learning that explores the geometry and distribution of embeddings. In this way, we mitigate the uneven performance over image classes caused by data imbalance, and suppress the spurious correlation between the input images and output predictions such that the model can be better generalized to new classes and maintain robust accuracy with a small number of labeled samples as reference. Then, we extend our analysis to the open-text description of each sample and explore the noisy label in multi-modal pre-training. We build our framework upon contrastive language-image pretraining to learn a common representation space and improve the training effectiveness by automatically eliminating false negative labels and correcting the false positives. Additionally, our approaches show the potential to tackle the label bias in multi-modal training data. Throughout this dissertation, the unifying focus is on the effective approach for learning from low-quality data, which has considered the learning issues from two complementary aspects of data labeling, i.e., the bias in global distribution and the noise in annotation for each sample (local). Different from prior research that are developed on the data with biased & noisy label but artificially simulated from well-curated datasets, our approach has been validated to be resilient to the complex bias and noise in the real-world scenario. We hope our approach can offer contributions to the field of multi-modal machine learning with applications involving real-world low-quality data and the need to avoid manual effort in data construction.

No description available.

Visual recognition tasks have witnessed vast improvements in performance since the advent of deep learning. Despite the gains in performance, image understanding algorithms are still not completely robust to partial occlusion. In this work, we propose a novel object classification method based on compositional modeling and explore its effect in the context of the newly introduced panoptic segmentation task. The panoptic segmentation task combines both semantic and instance segmentation to perform labelling of the entire image. The novel classification method replaces the object detection pipeline in UPSNet, a Mask R-CNN based design for panoptic segmentation. We also discuss an issue with the segmentation mask prediction of Mask R-CNN that affects overlapping instances. We perform extensive experiments and showcase results on the complex COCO and Cityscapes datasets. The novel classification method shows promising results for object classification on occluded instances in complex scenes. / Master of Science / Visual recognition tasks have witnessed vast improvements in performance since the advent of deep learning. Despite making significant improvements, algorithms for these tasks still do not perform well at recognizing partially visible objects in the scene. In this work, we propose a novel object classification method that uses compositional models to perform part based detection. The method first looks at individual parts of an object in the scene and then makes a decision about its identity. We test the proposed method in the context of the recently introduced panoptic segmentation task. The panoptic segmentation task combines both semantic and instance segmentation to perform labelling of the entire image. The novel classification method replaces the object detection module in UPSNet, a Mask R-CNN based algorithm for panoptic segmentation. We also discuss an issue with the segmentation mask prediction of Mask R-CNN that affects overlapping instances. After performing extensive experiments and evaluation, it can be seen that the novel classification method shows promising results for object classification on occluded instances in complex scenes.

Contemporary machine learning models exhibit unprecedented performance in the text, vision, and time-series domains, but at the cost of significant computational and human resources. Applying these technologies for science requires balancing accuracy and resource allocation, which I investigate here via three unique case studies. In Chapter 1, I present a deep learning system for animal pose estimation from video. Existing approaches rely on frame-by-frame supervised deep learning, which requires extensive manual labeling, fails to generalize to data far outside of its training set, and occasionally produces scientifically-critical errors that are hard to detect. The solution proposed here includes semi-supervised learning on unlabeled videos, video-centric network architectures, and a post-processing step that combines network ensembling and state-space modeling. These methods improve performance both with scarce and abundant labels, and are implemented in an easy-to-use software package and cloud application. In Chapter 2, I turn to the Gaussian process, a canonical nonparametric model, known for its poor scaling with dataset size. Existing methods accelerate Gaussian processes at the cost of modeling biases. I analyze two common techniques -- early truncated conjugate gradients and random Fourier features -- showing that they find hyperparameters that underfit and overfit the data, respectively. I then propose to eliminate these biases in exchange of increased variance, via randomized truncation estimators. In In Chapter 3, I investigate continual learning, or "finetuning", in large language models (LLMs) with billions of weights. Training these models requires more memory than typically available in academic clusters. Low-Rank Adaptation (LoRA) is a widely-used technique that saves memory by training only low rank perturbations to selected weight matrices in a so-called "base model'". I compare the performance of LoRA and full finetuning on two target domains, programming and mathematics, across different data regimes. I find that in most common settings, LoRA underperforms full finetuning, but it nevertheless exhibits a desirable form of regularization: it better maintains the base model's performance on tasks outside the target domain. I then propose best practices for finetuning with LoRA. In summary, applying state-of-the-art models to large scientific datasets necessitates taking computational shortcuts. This thesis highlights the implications of these shortcuts and emphasizes the need for careful empirical and theoretical investigation to find favorable trade-offs between accuracy and resource allocation.