Indiana University-Purdue University Indianapolis (IUPUI) / Medication non-adherence and non-compliance left unaddressed can compound into severe medical problems for patients. Identifying patients that are likely to become non-adherent can help reduce these problems. Despite these benefits, monitoring adherence at scale is cost-prohibitive. Social forums offer an easily accessible, affordable, and timely alternative to the traditional methods based on claims data. This study investigates the potential of medication adherence prediction based on social forum data for diabetes and fibromyalgia therapies by using transfer learning from the Medical Expenditure Panel Survey (MEPS). Predictive adherence models are developed by using both survey and social forums data and different random forest (RF) techniques. The first of these implementations uses binned inputs from k-means clustering. The second technique is based on ternary trees instead of the widely used binary decision trees. These techniques are able to handle missing data, a prevalent characteristic of social forums data. The results of this study show that transfer learning between survey models and social forum models is possible. Using MEPS survey data and the techniques listed above to derive RF models, less than 5% difference in accuracy was observed between the MEPS test dataset and the social forum test dataset. Along with these RF techniques, another RF implementation with imputed means for the missing values was developed and shown to predict adherence for social forum patients with an accuracy >70%. This thesis shows that a model trained with verified survey data can be used to complement traditional medical adherence models by predicting adherence from unverified, self-reported data in a dynamic and timely manner. Furthermore, this model provides a method for discovering objective insights from subjective social reports. Additional investigation is needed to improve the prediction accuracy of the proposed model and to assess biases that may be inherent to self-reported adherence measures in social health networks.

京都大学 / 新制・課程博士 / 博士(工学) / 甲第23615号 / 工博第4936号 / 新制||工||1771(附属図書館) / 京都大学大学院工学研究科機械理工学専攻 / (主査)教授 椹木 哲夫, 教授 松野 文俊, 教授 藤本 健治 / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DFAM

The introduction of Machine Learning (ML) and Deep Learning (DL) techniques into modern radio communications system, a field known as Radio Frequency Machine Learning (RFML), has the potential to provide increased performance and flexibility when compared to traditional signal processing techniques and has broad utility in both the commercial and defense sectors. Existing RFML systems predominately utilize supervised learning solutions in which the training process is performed offline, before deployment, and the learned model remains fixed once deployed. The inflexibility of these systems means that, while they are appropriate for the conditions assumed during offline training, they show limited adaptability to changes in the propagation environment and transmitter/receiver hardware, leading to significant performance degradation. Given the fluidity of modern communication environments, this rigidness has limited the widespread adoption of RFML solutions to date. Transfer Learning (TL) is a means to mitigate such performance degradations by re-using prior knowledge learned from a source domain and task to improve performance on a "similar" target domain and task. However, the benefits of TL have yet to be fully demonstrated and integrated into RFML systems. This dissertation begins by clearly defining the problem space of RF TL through a domain-specific TL taxonomy for RFML that provides common language and terminology with concrete and Radio Frequency (RF)-specific example use- cases. Then, the impacts of the RF domain, characterized by the hardware and channel environment(s), and task, characterized by the application(s) being addressed, on performance are studied, and methods and metrics for predicting and quantifying RF TL performance are examined. In total, this work provides the foundational knowledge to more reliably use TL approaches in RF contexts and opens directions for future work that will improve the robustness and increase the deployability of RFML. / Doctor of Philosophy / The field of Radio Frequency Machine Learning (RFML) introduces Machine Learning (ML) and Deep Learning (DL) techniques into modern radio communications systems, and is expected to be a core component of 6G technologies and beyond. While RFML provides a myriad of benefits over traditional radio communications systems, existing approaches are generally incapable of adapting to changes that will inevitably occur over time, which causes severe performance degradation. Transfer Learning (TL) offers a solution to the inflexibility of current RFML systems, through techniques for re-using and adapting existing models for new, but similar, problems. TL is an approach often used in image and language-based ML/DL systems, but has yet to be commonly used by RFML researchers. This dissertation aims to provide the foundational knowledge necessary to reliably use TL in RFML systems, from the definition and categorization of RF TL techniques to practical guidelines for when to use RF TL in real-world systems. The unique elements of RF TL not present in other modalities are exhaustively studied, and methods and metrics for measuring and predicting RF TL performance are examined.

Retrogressive thaw slumps (RTSs) are thermokarst landforms that develop on slopes in permafrost regions when thawing permafrost causes the land surface to collapse. RTSs are an indicator of climate change and pose a threat to human infrastructure and ecosystems in the affected areas. As the availability of ready-to-use high-resolution satellite imagery increases, automatic RTS mapping is being explored with deep learning methods. We employed a pre-trained Mask-RCNN model to automatically map RTSs on Banks Island and Victoria Island in the western Canadian Arctic, where there is extensive RTS activity. We tested the model with different settings, including image band combinations, backbones, and backbone trainable layers, and performed hyper-parameter tuning and determined the optimal learning rate, momentum, and decay rate for each of the model settings. Our final model successfully mapped most of the RTSs in our test sites, with F1 scores ranging from 0.61 to 0.79. Our study demonstrates that transfer learning from a pre-trained Mask-RCNN model is an effective approach that has the potential to be applied for RTS mapping across the Canadian Arctic.

Deep learning is increasingly becoming a viable way of classifying all types of data. Modern deep learning algorithms, such as one dimensional convolutional neural networks, have demonstrated excellent performance in classifying time series data because of the ability to identify time invariant features. A primary challenge of deep learning for time series classification is the large amount of data required for training and many application domains, such as in medicine, have challenges obtaining sufficient data. Transfer learning is a deep learning method used to apply feature knowledge from one deep learning model to another; this is a powerful tool when both training datasets are similar and offers smaller datasets the power of more robust larger datasets. This makes it vital that the best source dataset is selected when performing transfer learning and presently there is no metric for this purpose. In this thesis a metric of predicting the performance of transfer learning is proposed. To develop this metric this research will focus on classification and transfer learning for human-activity-recognition time series data. For general time series data, finding temporal relations between signals is computationally intensive using non-deep learning techniques. Rather than time-series signal processing, a neural network autoencoder was used to first transform the source and target datasets into a time independent feature space. To compare and quantify the suitability of transfer learning datasets, two metrics were examined: i) average embedded signal from each dataset was used to calculate the distance between each datasets centroid, and ii) a Generative Adversarial Network (GAN) model was trained and the discriminator portion of the GAN is then used to assess the dissimilarity between source and target. This thesis measures a correlation between the distance between two dataset and their similarity, as well as the ability for a GAN to discriminate between two datasets and their similarity. The discriminator metric, however, does suffer from an upper limit of dissimilarity. These metrics were then used to predict the success of transfer learning from one dataset to another for the purpose of general time series classification. / Thesis / Master of Applied Science (MASc) / Over the past decade, advances in computational power and increases in data quantity have made deep learning a useful method of complex pattern recognition and classification in data. There is a growing desire to be able to use these complex algorithms on smaller quantities of data. To achieve this, a deep learning model is first trained on a larger dataset and then retrained on the smaller dataset; this is called transfer learning. For transfer learning to be effective, there needs to be a level of similarity between the two datasets so that properties from larger dataset can be learned and then refined using the smaller dataset. Therefore, it is of great interest to understand what level of similarity exists between the two datasets. The goal of this research is to provide a similarity metric between two time series classification datasets so that potential performance gains from transfer learning can be better understood. The measure of similarity between two time series datasets presents a unique challenge due to the nature of this data. To address this challenge an encoder approach was implemented to transform the time series data into a form where each signal example can be compared against one another. In this thesis, different similarity metrics were evaluated and correlated to the performance of a deep learning model allowing the prediction of how effective transfer learning may be when applied.

The objective of this dissertation is to provide key methodological advancements towards the use of transfer learning in Laser-Based Additive Manufacturing (LBAM), to assist practitioners in producing high-quality repeatable parts. Currently, in LBAM processes, there is an urgent need to improve the quality and repeatability of the manufacturing process. Fabricating parts using LBAM is often expensive, due to the high cost of materials, the skilled machine operators needed for operation, and the long build times needed to fabricate parts. Additionally, monitoring the LBAM process is expensive, due to the highly specialized infrared sensors needed to monitor the thermal evolution of the part. These factors lead to a key challenge of improving the quality of additively manufactured parts, because additional experiments and/or sensors is expensive. We propose to use transfer learning, which is a statistical technique for transferring knowledge from one domain to a similar, yet distinct, domain, to leverage previous non-identical experiments to assist practitioners in expediting part certification. By using transfer learning, previous experiments completed in similar, but non-identical, domains can be used to provide insight towards the fabrication of high-quality parts. In this dissertation, transfer learning is applied to four key domains within LBAM. First, transfer learning is used for sensor fusion, specifically to calibrate the infrared camera with true temperature measurements from the pyrometer. Second, a Bayesian transfer learning approach is developed to transfer knowledge across different material systems, by modelling material differences as a lurking variable. Third, a Bayesian transfer learning approach for predicting distortion is developed to transfer knowledge from a baseline machine system to a new machine system, by modelling machine differences as a lurking variable. Finally, compensation plans are developed from the transfer learning models to assist practitioners in improving the quality of parts using previous experiments. The work of this dissertation provides current practitioners with methods for sensor fusion, material/machine calibration, and efficient learning of compensation plans with few samples.

<p dir="ltr">Despite the remarkable advancements of machine learning (ML) technologies in biomedical research, especially in tackling complex human diseases such as cancer and Alzheimer's disease, a considerable gap persists between promising theoretical results and dependable clinical applications in diagnosis, prognosis, and therapeutic decision-making. One of the primary challenges stems from the absence of large high-quality patient datasets, which arises from the cost and human labor required for collecting such datasets and the scarcity of patient samples. Moreover, the inherent complexity of the data often leads to a feature space dimension that is large compared with the sample size, potentially causing instability during training and unreliability in inference. To address these challenges, the transfer learning (TL) approach has been embraced in biomedical ML applications to facilitate knowledge transfer across diverse and related biological contexts. Leveraging this principle, we introduce an unsupervised multi-view TL algorithm, named MVTOT [1], which enables the analysis of various biomarkers across different cancer types. Specifically, we compress high-dimensional biomarkers from different cancer types into a low-dimensional feature space via nonnegative matrix factorization and distill common information shared by various cancer types using the Wasserstein distance defined by Optimal Transport theory. We evaluate the stratification performance on three early-stage cancers from the Cancer Genome Atlas (TCGA) project. Our framework, compared with other benchmark methods, demonstrates superior accuracy in patient survival outcome stratification.</p><p dir="ltr">Additionally, while patient-level stratification has enhanced clinical decision-making, our understanding of diseases at the single-cell (SC) level remains limited, which is crucial for deciphering disease progression mechanisms, monitoring drug responses, and prioritizing drug targets. It is essential to associate each SC with patient-level clinical traits such as survival hazard, drug response, and disease subtypes. However, SC samples often lack direct labeling with these traits, and the significant statistical gap between patient and SC-level gene expressions impedes the transfer of well-annotated patient-level disease attributes to SCs. Domain adaptation (DA), a TL subfield, addresses this challenge by training a domain-invariant feature extractor for both patient and SC gene expression matrices, facilitating the successful application of ML models trained on patient-level data to SC samples. Expanding upon an established deep-learning-based DA model, DEGAS [2], we substitute their computationally ineffective maximum mean discrepancy loss with the Wasserstein distance as the metric for domain discrepancy. This substitution facilitates the embedding of both SC and patient inputs into a common latent feature space. Subsequently, employing the model trained on patient-level disease attributes, we predict SC-level survival hazard, disease status, and drug response for prostate cancer, Alzheimer's SC data, and multiple myeloma data, respectively. Our approach outperforms benchmark studies, uncovering clinically significant cell subgroups and revealing the correlation between survival hazard and drug response at the SC level.</p><p dir="ltr">Furthermore, in addition to these approaches, we acknowledge the effectiveness of TL and image analysis in stratifying patients with early and late-stage Mild Cognitive Impairment based on neuroimaging, as well as predicting survival and metastasis in melanoma based on histological images. These applications underscore the potential of employing ML methods, especially TL algorithms, in addressing biomedical issues from various angles, thereby enhancing our understanding of disease mechanisms and developing new biomarkers predicting patient outcomes.</p>

Radio frequency fingerprinting, a type of physical layer identification, allows identifying wireless transmitters based on their unique hardware. Every wireless transmitter has slight manufacturing variations and differences due to the layout of components. These are manifested as differences in the signal emitted by the device. A variety of techniques have been proposed for identifying transmitters, at the physical layer, based on these differences. This has been successfully demonstrated on a large variety of transmitters and other devices. However, some situations still pose challenges: Some types of fingerprinting feature are very dependent on the modulated signal, especially features based on the frequency content of a signal. This means that changes in transmitter configuration such as bandwidth or modulation will prevent wireless fingerprinting. Such changes may occur frequently with cognitive radios, and in dynamic spectrum access networks. A method is proposed to transform features to be invariant with respect to changes in transmitter configuration. With the transformed features it is possible to re-identify devices with a high degree of certainty. Next, improving performance with limited data by identifying devices using observations crowdsourced from multiple receivers is examined. Combinations of three types of observations are defined. These are combinations of fingerprinter output, features extracted from multiple signals, and raw observations of multiple signals. Performance is demonstrated, although the best method is dependent on the feature set. Other considerations are considered, including processing power and the amount of data needed. Finally, drift in fingerprinting features caused by changes in temperature is examined. Drift results from gradual changes in the physical layer behavior of transmitters, and can have a substantial negative impact on fingerprinting. Even small changes in temperature are found to cause drift, with the oscillator as the primary source of this drift (and other variation) in the fingerprints used. Various methods are tested to compensate for these changes. It is shown that frequency based features not dependent on the carrier are unaffected by drift, but are not able to distinguish between devices. Several models are examined which can improve performance when drift is present. / Doctor of Philosophy / Radio frequency fingerprinting allows uniquely identifying a transmitter based on characteristics of the signal it emits. In this dissertation several extensions to current fingerprinting techniques are given. Together, these allow identification of transmitters which have changed the signal sent, identifying using different measurement types, and compensating for variation in a transmitter's behavior due to changes in temperature.

Numerous of high-impact applications involve predictive modeling of real-world data. This spans from hospital readmission prediction for enhanced patient care up to event detection in power systems for grid stabilization. Developing performant machine learning models necessitates extensive high-quality training data, ample labeled samples, and training and testing datasets derived from identical distributions. Though, such methodologies may be impractical in applications where obtaining labeled data is expensive or challenging, the quality of data is low, or when challenged with covariate or concept shifts. Our emphasis was on devising transfer learning methods to address the inherent challenges across two distinct applications.We delved into a notably challenging transfer learning application that revolves around predicting hospital readmission risks using electronic health record (EHR) data to identify patients who may benefit from extra care. Readmission models based on EHR data can be compromised by quality variations due to manual data input methods. Utilizing high-quality EHR data from a different hospital system to enhance prediction on a target hospital using traditional approaches might bias the dataset if distributions of the source and target data are different. To address this, we introduce an Early Readmission Risk Temporal Deep Adaptation Network, ERR-TDAN, for cross-domain knowledge transfer. A model developed using target data from an urban academic hospital was enhanced by transferring knowledge from high-quality source data. Given the success of our method in learning from data sourced from multiple hospital systems with different distributions, we further addressed the challenge and infeasibility of developing hospital-specific readmission risk prediction models using data from individual hospital systems. Herein, based on an extension of the previous method, we introduce an Early Readmission Risk Domain Generalization Network, ERR-DGN. It is adept at generalizing across multiple EHR data sources and seamlessly adapting to previously unseen test domains. In another challenging application, we addressed event detection in electrical grids where dependencies are spatiotemporal, highly non-linear, and non-linear systems using high-volume field-recorded data from multiple Phasor Measurement Units (PMUs). Existing historical event logs created manually do not correlate well with the corresponding PMU measurements due to scarce and temporally imprecise labels. Extending event logs to a more complete set of labeled events is very costly and often infeasible to obtain. We focused on utilizing a transfer learning method tailored for event detection from PMU data to reduce the need for additional manual labeling. To demonstrate the feasibility, we tested our approach on large datasets collected from the Western and Eastern Interconnections of the U.S.A. by reusing a small number of carefully selected labeled PMU data from a power system to detect events from another. Experimental findings suggest that the proposed knowledge transfer methods for healthcare and power system applications have the potential to effectively address the identified challenges and limitations. Evaluation of the proposed readmission models show that readmission risk predictions can be enhanced when leveraging higher-quality EHR data from a different site, and when trained on data from multiple sites and subsequently applied to a novel hospital site. Moreover, labels scarcity in power systems can be addressed by a transfer learning method in conjunction with a semi-supervised algorithm that is capable of detecting events based on minimal labeled instances. / Computer and Information Science

Deep neural networks can achieve state-of-the-art performance on various image recognition tasks, such as object categorization (image classification) and object localization (object detection), with the help of a large amount of training data. However, to achieve models that perform well in the real world, we must overcome the shift from training to real-world data, which involves two factors: (1) covariate shift and (2) unseen classes. Covariate shift occurs when the input distribution of a particular category changes from the training time. Deep models can easily make mistakes with a small change in the input, such as small noise addition, lighting change, or changes in the object pose. On the other hand, unseen classes - classes that are absent in the training set - may be present in real-world test samples. It is important to differentiate between "seen" and "unseen" classes in image classification, while locating diverse classes, including classes unseen during training, is crucial in object detection. Therefore, an open-world image recognition model needs to handle both factors. In this thesis, we propose approaches for image classification and object detection that can handle these two kinds of shifts in a label-efficient way. Firstly, we examine the adaptation of large-scale pre-trained models to the object detection task while preserving their robustness to handle covariate shift. We investigate various pre-trained models and discover that the acquisition of robust representations by a trained model depends heavily on the pre-trained model’s architecture. Based on this intuition, we develop simple techniques to prevent the loss of generalizable representations. Secondly, we study the adaptation to an unlabeled target domain for object detection to address the covariate shift. Traditional domain alignment methods may be inadequate due to various factors that cause domain shift between the source and target domains, such as layout and the number of objects in an image. To address this, we propose a strong-weak distribution alignment approach that can handle diverse domain shifts. Furthermore, we study the problem of semi-supervised domain adaptation for image classification when partially labeled target data is available. We introduce a simple yet effective approach, MME, for this task, which extracts discriminative features for the target domain using adversarial learning. We also develop a method to handle the situation where the unlabeled target domain includes categories unseen in the source domain. Since there is no supervision, recognizing instances of unseen classes as "unseen" is challenging. To address this, we devise a straightforward approach that trains a one-vs-all classifier using source data to build a classifier that can detect unseen instances. Additionally, we introduce an approach to enable an object detector to recognize an unseen foreground instance as an "object" using a simple data augmentation and learning framework that is applicable to diverse detectors and datasets. In conclusion, our proposed approaches employ various datasets or architectures due to their simple design and achieve state-of-the-art results. Our work can contribute to the development of a unified open-world image recognition model in future research.