Image representation is a key component in visual recognition systems. In visual recognition problem, the solution or the model should be able to learn and infer the quality of certain visual semantics in the image. Therefore, it is important for the model to represent the input image in a way that the semantics of interest can be inferred easily and reliably. This thesis is written in the form of a compilation of publications and tries to look into the Convolutional Networks (CovnNets) representation in visual recognition problems from an empirical perspective. Convolutional Network is a special class of Neural Networks with a hierarchical structure where every layer’s output (except for the last layer) will be the input of another one. It was shown that ConvNets are powerful tools to learn a generic representation of an image. In this body of work, we first showed that this is indeed the case and ConvNet representation with a simple classifier can outperform highly-tuned pipelines based on hand-crafted features. To be precise, we first trained a ConvNet on a large dataset, then for every image in another task with a small dataset, we feedforward the image to the ConvNet and take the ConvNets activation on a certain layer as the image representation. Transferring the knowledge from the large dataset (source task) to the small dataset (target task) proved to be effective and outperformed baselines on a variety of tasks in visual recognition. We also evaluated the presence of spatial visual semantics in ConvNet representation and observed that ConvNet retains significant spatial information despite the fact that it has never been explicitly trained to preserve low-level semantics. We then tried to investigate the factors that affect the transferability of these representations. We studied various factors on a diverse set of visual recognition tasks and found a consistent correlation between the effect of those factors and the similarity of the target task to the source task. This intuition alongside the experimental results provides a guideline to improve the performance of visual recognition tasks using ConvNet features. Finally, we addressed the task of visual instance retrieval specifically as an example of how these simple intuitions can increase the performance of the target task massively. / <p>QC 20161209</p>

The phenomenon of transfer—our ability to perform novel tasks by generalizing from past experiences—has long captivated theorists and practitioners. As educators it is essential for us to understand what types of learning best promote transfer and to structure our curricula accordingly. With that goal in mind, this dissertation outlines two lines of research. For the first line of research I adopted an experimental approach in the domain of problem solving, examining a training technique whereby the learner solves practice problems for the same principle in dissimilar contexts as opposed to highly similar contexts. The key finding was that contextual variability improved transfer outcomes when a set of training problems were solved spaced in time (akin to a closed-book test), but not when prior training problems and their solutions remained visible throughout training (akin to an open-book test). This finding suggests that contextual variability during training can be beneficial because it forces the learner to more effortfully recall what they have learned in the past. For the second line of research I then adopted a correlational approach, investigating a ubiquitous self-report inventory, the Study Process Questionnaire (SPQ), which is meant to quantify student learning approaches to predict educational outcomes. However, the SPQ’s predictive validity has recently been challenged because deep learning and its corresponding outcomes remain poorly defined. To tackle this measurement issue, my colleagues and I operationally defined outcome measures in real university courses to tap more precisely into transfer of learning. Across several studies we found limited evidence for the SPQ’s ability to predict transfer outcomes, leading us to suggest that educators and researchers should be more cautious about using this self-report inventory to characterize student learning. / Thesis / Doctor of Philosophy (PhD) / A central goal of education is to equip students with ‘flexible’ knowledge, enabling them to transfer the far-reaching principles they have learned to solve new, real-world problems. But what conditions of training are most conducive to transfer? One understudied technique involves being tested on the same principle in dissimilar contexts. The experiments reported in Chapter 2 provide evidence for this training technique in the domain of problem solving. Aside from direct interventions, another approach has been to measure individual differences among students to predict how much they engage in “deep learning”—a process closely associated with transfer. However, four correlational studies in Chapters 3 and 4 revealed little support for this approach, highlighting the difficulty of characterizing learning strategies using self-reports. In sum, this shows promise for interventions involving repeated testing in dissimilar contexts, but little promise for a self-report inventory meant to capture individual differences in student learning.

Modern microscopy and automation technologies enable experiments which can produce millions of images each day. The valuable information is often sparse, and requires clever methods to find useful data. In this thesis a general image classification tool for fluorescence microscopy images was developed usingfeatures extracted from a general Convolutional Neural Network (CNN) trained on natural images. The user selects interesting regions in a microscopy image and then, through an iterative process, using active learning, continually builds a training data set to train a classifier that finds similar regions in other images. The classifier uses conformal prediction to find samples that, if labeled, would most improve the learned model as well as specifying the frequency of errors the classifier commits. The result show that with the appropriate choice of significance one can reach a high confidence in true positive. The active learning approach increased the precision with a downside of finding fewer examples.

Recognizing visual information is crucial for many real artificial-intelligence-based applications, ranging from domestic robots to autonomous vehicles. However, the success of deep learning methods on visual recognition tasks is highly dependent on access to large-scale labeled datasets, which are expensive and cumbersome to collect. Transfer learning provides a way to alleviate the burden of annotating data, which transfers the knowledge learned from a rich-labeled source domain to a scarce-labeled target domain. However, the performance of deep learning models degrades significantly when testing on novel domains due to the presence of domain shift. To tackle the domain shift, conventional domain adaptation methods diminish the domain shift between two domains with a distribution matching loss or adversarial loss. These models align the domain-specific feature distribution and the domain-invariant feature distribution simultaneously, which is sub-optimal towards solving deep domain adaptation tasks, given that deep neural networks are known to extract features in which multiple hidden factors are highly entangled. This thesis explores how to learn effective transferable features by disentangling the deep features. The following questions are studied: (1) how to disentangle the deep features into domain-invariant and domain-specific features? (2) how would feature disentanglement help to learn transferable features under a synthetic-to-real domain adaptation scenario? (3) how would feature disentanglement facilitate transfer learning with multiple source or target domains? (4) how to leverage feature disentanglement to boost the performance in a federated system? To address these needs, this thesis proposes deep adversarial feature disentanglement: a class/domain identifier is trained on the labeled source domain and the disentangler generates features to fool the class/domain identifier. Extensive experiments and empirical analysis demonstrate the effectiveness of the feature disentanglement method on many real-world domain adaptation tasks. Specifically, the following three unsupervised domain adaptation scenarios are explored: (1) domain agnostic learning with disentangled representations, (2) unsupervised federated domain adaptation, (3) multi-source domain adaptation.

Up to 87% of the world’s wetlands have been destroyed, considerably reducing ecosystem services these wetlands once provided. More recently, many wetlands are being restored in an attempt to regain their ecosystem service. This study seeks to determine the effects of restored wetlands on local bat habitat use. Bat activity was found to be significantly higher around the wetlands when compared to distant grassy fields; however, no significant difference was found among the restored wetlands and a remote cattle farm containing multiple water features. Geospatial models of bat distribution and bat foraging were produced using machine learning that showed higher habitat suitability and foraging activity around restored wetlands than around distant grassy fields, suggesting that wetlands provide vital habitat for insectivorous bats. This study demonstrates that restored wetlands promote bat activity and bat foraging, and restoring wetlands may be a useful means of increasing natural pest control over nearby farmlands.

In this work, we develop the transfer learning (TL) of reinforcement learning (RL) for the robotic skill of throwing a ball into a basket, from a computer simulated environment to a real-world implementation. Whereas learning of the same skill has been previously explored by using a Programming by Demonstration approach directly on the real-world robot, for our work, the model-based RL algorithm PILCO was employed as an alternative as it provides the robot with no previous knowledge or hints, i.e. the robot begins learning from a tabula rasa state, PILCO learns directly on the simulated environment, and as part of its procedure, PILCO models the dynamics of the inflatable, plastic ball used to perform the task. The robotic skill is represented as a Markov Decision Process, the robotic arm is a Kuka LWR4+, RL is enabled by PILCO, and TL is achieved through policy adjustments. Two learned policies were transferred, and although the results show that no exhaustive policy adjustments are required, large gaps remain between the simulated and the real environment in terms of the ball and robot dynamics. The contributions of this thesis include: a novel TL of RL framework for teaching the basketball skill to the Kuka robotic arm; the development of a pythonised version of PILCO; robust and extendable ROS packages for policy learning and adjustment in a simulated or real robot; a tracking-vision package with a Kinect camera; and an Orocos package for a position controller in the robotic arm.

Indiana University-Purdue University Indianapolis (IUPUI) / Domain adaptation is one of the hottest directions in solving annotation insufficiency problem of deep learning. General domain adaptation is not consistent with the practical scenarios in the industry. In this thesis, we focus on two concerns as below. First is that labeled data are generally collected from multiple domains. In other words, multi-source adaptation is a more common situation. Simply extending these single-source approaches to the multi-source cases could cause sub-optimal inference, so specialized multi-source adaptation methods are essential. The main challenge in the multi-source scenario is a more complex divergence situation. Not only the divergence between target and each source plays a role, but the divergences among distinct sources matter as well. However, the significance of maintaining consistency among multiple sources didn't gain enough attention in previous work. In this thesis, we propose an Enhanced Consistency Multi-Source Adaptation (EC-MSA) framework to address it from three perspectives. First, we mitigate feature-level discrepancy by cross-domain conditional alignment, narrowing the divergence between each source and target domain class-wisely. Second, we enhance multi-source consistency via dual mix-up, diminishing the disagreements among different sources. Third, we deploy a target distilling mechanism to handle the uncertainty of target prediction, aiming to provide high-quality pseudo-labeled target samples to benefit the previous two aspects. Extensive experiments are conducted on several common benchmark datasets and demonstrate that our model outperforms the state-of-the-art methods. Second is that data privacy and security is necessary in practice. That is, we hope to keep the raw data stored locally while can still obtain a satisfied model. In such a case, the risk of data leakage greatly decreases. Therefore, it is natural for us to combine the federated learning paradigm with domain adaptation. Under the source-private setting, the main challenge for us is to expose information from the source domain to the target domain while make sure that the communication process is safe enough. In this thesis, we propose a method named Fourier Transform-Assisted Federated Domain Adaptation (FTA-FDA) to alleviate the difficulties in two ways. We apply Fast Fourier Transform to the raw data and transfer only the amplitude spectra during the communication. Then frequency space interpolations between these two domains are conducted, minimizing the discrepancies while ensuring the contact of them and keeping raw data safe. What's more, we make prototype alignments by using the model weights together with target features, trying to reduce the discrepancy in the class level. Experiments on Office-31 demonstrate the effectiveness and competitiveness of our approach, and further analyses prove that our algorithm can help protect privacy and security.

Statistics reveal a huge increase in cyberattacks making technology businesses more susceptible to data loss. With increasing application of machine learning in different domains, studies have been focused on building cognitive models for traffic anomaly detection in a communication network. These studies have led to generation of datasets containing network traffic data packets, usually captured using softwares like Wireshark. These datasets contain high dimensional data corresponding to benign data packets and attack data packets of known attacks. Recent research has mainly focused on developing machine learning architectures that are able to extract useful information from high dimensional datasets to detect attack data packets in a network. In addition, machine learning algorithms are currently trained to detect only documented attacks with available training data. However, with the proliferation of new cyberattacks and zero-day attacks with little to no training data available, current employed algorithms have little to no success in detecting new attacks. In this thesis, we focus on detecting rare attacks using transfer learning from a dataset containing information pertaining to known attacks.<div><br></div><div>In the literature, there is proof of concept for both classical machine learning and deep learning approaches for anomaly detection. We show that a deep learning approach outperforms explicit statistical modeling based approaches by at least 21% for the used dataset. We perform a preliminary survey of candidate deep learning architectures before testing for transferability and propose a Convolutional Neural Network architecture that is 99.65% accurate in classifying attack data packets.<br></div><div><br></div><div>To test for transferability, we train this proposed CNN architecture with a known attack and test it's performance on attacks that are unknown to the network. For this model to extract adequate information for transferability, the model requires a higher representation of attack data in the training dataset with the current attack data comprising only 20% of the dataset. To overcome the problem of small training sets, several techniques to boost the number of attack data packets are employed like a novel synthetic dataset based training and bootstrapped dataset training.<br></div><div><br></div><div>Our study results in identification of training-testing attack pairs that show high learning transferability. Most of the strong and consistent correlations are observed among Denial of Service(DoS) training-testing attack pairs. Furthermore, we propose hypotheses for model generalization. Our results are validated by a study of dataset features and attack characteristics using the Recursive Feature Elimination(RFE) algorithm. <br></div>

This empirical research study discusses how much the model’s accuracy changes when adding a new image class by using a pre-trained model with the same labels and measuring the precision of the previous classes to observe the changes. The purpose is to determine if using transfer learning is beneficial for users that do not have enough data to train a model. The pre-trained model that was used to create a new model was the Inception V3. It has the same labels as the eight different classes that were used to train the model. To test this model, classes of wild and non-wild animals were taken as samples. The algorithm used to train the model was implemented in a single class programmed in Python programming language with PyTorch and TensorBoard library. The Tensorboard library was used to collect and represent the result. Research results showed that the accuracy of the first two classes was 94.96% in training and 97.07% in validation. When training the model with a total of eight classes, the accuracy was 91.89% in training and 95.40 in validation. The precision of both classes was detected at 100% when the model solely had cat and dog classes. After adding six additional classes in the model, the precision changed to 95.82% of the cats and 97.16% of the dogs.

Radio frequency fingerprinting, a type of physical layer identification, allows identifying wireless transmitters based on their unique hardware. Every wireless transmitter has slight manufacturing variations and differences due to the layout of components. These are manifested as differences in the signal emitted by the device. A variety of techniques have been proposed for identifying transmitters, at the physical layer, based on these differences. This has been successfully demonstrated on a large variety of transmitters and other devices. However, some situations still pose challenges: Some types of fingerprinting feature are very dependent on the modulated signal, especially features based on the frequency content of a signal. This means that changes in transmitter configuration such as bandwidth or modulation will prevent wireless fingerprinting. Such changes may occur frequently with cognitive radios, and in dynamic spectrum access networks. A method is proposed to transform features to be invariant with respect to changes in transmitter configuration. With the transformed features it is possible to re-identify devices with a high degree of certainty. Next, improving performance with limited data by identifying devices using observations crowdsourced from multiple receivers is examined. Combinations of three types of observations are defined. These are combinations of fingerprinter output, features extracted from multiple signals, and raw observations of multiple signals. Performance is demonstrated, although the best method is dependent on the feature set. Other considerations are considered, including processing power and the amount of data needed. Finally, drift in fingerprinting features caused by changes in temperature is examined. Drift results from gradual changes in the physical layer behavior of transmitters, and can have a substantial negative impact on fingerprinting. Even small changes in temperature are found to cause drift, with the oscillator as the primary source of this drift (and other variation) in the fingerprints used. Various methods are tested to compensate for these changes. It is shown that frequency based features not dependent on the carrier are unaffected by drift, but are not able to distinguish between devices. Several models are examined which can improve performance when drift is present. / Doctor of Philosophy / Radio frequency fingerprinting allows uniquely identifying a transmitter based on characteristics of the signal it emits. In this dissertation several extensions to current fingerprinting techniques are given. Together, these allow identification of transmitters which have changed the signal sent, identifying using different measurement types, and compensating for variation in a transmitter's behavior due to changes in temperature.