At present, high-dimensional data sets are becoming more and more frequent. The problem of feature selection has already become widespread, owing to the curse of dimensionality. Unfortunately, feature selection is largely based on ground truth and domain expertise. It is possible that ground truth and/or domain expertise will be unavailable, therefore there is a growing need for unsupervised feature selection in multiple fields, such as marketing and proteomics.Now, unlike in past time, it is possible for biologists to measure the amount of protein in a cancer cell. No wonder the data is high-dimensional, the human body is composed of thousands and thousands of proteins. Intuitively, only a handful of proteins cause the onset of the disease. It might be desirable to cluster the cancer sufferers, but at the same time we want to find the proteins that produce good partitions.We hereby propose a methodology designed to find the features able to maximize the clustering performance. After we divided the proteins into different groups, we clustered the patients. Next, we evaluated the clustering performance. We developed a couple of pipelines. Whilst the first focuses its attention on the data provided by the laboratory, the second takes advantage both of the external data on protein complexes and of the internal data. We set the threshold of clustering performance thanks to the biologists at Karolinska Institutet who contributed to the project.In the thesis we show how to make a good selection of features without domain expertise in case of breast cancer data. This experiment illustrates how we can reach a clustering performance up to eight times better than the baseline with the aid of feature selection. / Högdimensionella dataseter blir allt vanligare. Problemet med funktionsval har redan blivit utbrett på grund av dimensionalitetens förbannelse. Dessvärre är funktionsvalet i stor utsträckning baserat på grundläggande sanning och domänkunskap. Det är möjligt att grundläggande sanning och/eller domänkunskap kommer att vara otillgänglig, därför finns det ett växande behov av icke-övervakat funktionsval i flera områden, såsom marknadsföring och proteomics.I nuläge, till skillnad från tidigare, är det möjligt för biologer att mäta mängden protein i en cancercell. Inte undra på att data är högdimensionella, människokroppen består av tusentals och tusentals proteiner. Intuitivt orsakar bara en handfull proteiner sjukdomsuppkomsten. Det kan vara önskvärt att klustrera cancerlidarna, men samtidigt vill vi hitta proteiner som producerar goda partitioner.Vi föreslår härmed en metod som är utformad för att hitta funktioner som kan maximera klustringsprestandan. Efter att vi delat proteinerna i olika grupper klustrade vi patienterna. Därefter utvärderade vi klustringsprestandan. Vi utvecklade ett par pipelines. Medan den första fokuserar på de data som laboratoriet tillhandahåller, utnyttjar den andra både extern data på proteinkomplex och intern data. Vi ställde gränsen för klusterprestationen tack vare biologerna vid Karolinska Institutet som bidragit till projektet.I avhandlingen visar vi hur man gör ett bra utbud av funktioner utan domänkompetens vid bröstcancerdata. Detta experiment illustrerar hur vi kan nå en klusterprestation upp till åtta gånger bättre än baslinjen med hjälp av funktionsval.

The aim of this study is to research the possibility of using customer transactional data to identify spending patterns among individuals, that in turn can be used to assess creditworthiness. Two different approaches to unsupervised clustering are used and compared in the study, one being K-means and the other an hierarchical approach. The features used in both clustering techniques are extracted from customer transactional data collected from the customers banks. Internal cluster validity indices and credit scores, calculated by credit institutes, are used to evaluate the results of the clustering techniques. Based on the experiments in this report, we believe that the approach exhibit interesting results and that further research with evaluation on a larger dataset is desired. Proposed future work is to append additional features to the models and study the effect on the resulting clusters. / Målet med detta arbete är att studera möjligheten att använda data om individers kontotransaktioner för att identifiera utgiftsmönster hos individer, som i sin tur kan användas för att utvärdera kreditvärdighet. Två olika tillvägagångssätt som använder oövervakad klustring (eng. unsupervised clustering) används och utvärderas i rapporten, den ena är K-means och den andra är en hierarkisk teknik. De attribut (eng. features) som används i de båda klustrings teknikerna utvinns från data som innehåller kontotransaktioner och som erhålls från banker. Interna kluster värde index (eng. cluster validity indices) och individers riskprognoser, som beräknats av ett kreditinstitut, används för att utvärdera resultaten från klustrings teknikerna. Vi menar att resultaten som presenteras i denna rapport visar att målet till viss del uppnåtts, men att mer data och forskning krävs. Vidare forskning som föreslås är att lägga till fler attribut (eng. features) till modellerna och utvärdera effekten på de resulterande klusterna.

京都大学 / 新制・課程博士 / 博士(人間健康科学) / 甲第23825号 / 人健博第96号 / 新制||人健||7(附属図書館) / 京都大学大学院医学研究科人間健康科学系専攻 / (主査)教授 椎名 毅, 教授 精山 明敏, 教授 中本 裕士 / 学位規則第4条第1項該当 / Doctor of Human Health Sciences / Kyoto University / DFAM

Convolutional neural networks (CNNs) have dominated the computer vision field since the early 2010s, when deep learning largely replaced previous approaches like hand-crafted feature engineering and hierarchical image parsing. Meanwhile transformer architectures have attained preeminence in natural language processing, and have even begun to supplant CNNs as the state of the art for some computer vision tasks. This study proposes a novel transformer-based architecture, the attentional parsing network, that reconciles the deep learning and hierarchical image parsing approaches to computer vision. We recast unsupervised image representation as a sequence-to-sequence translation problem where image patches are mapped to successive layers of latent variables; and we enforce symmetry and sparsity constraints to encourage these mappings take the form of a parse tree. We measure the quality of learned representations by passing them to a classifier and find high accuracy (> 90%) for even small models. We also demonstrate controllable image generation: first by “back translating” from latent variables to pixels, and then by selecting subsets of those variables with attention masks. Finally we discuss our design choices and compare them with alternatives, suggesting best practices and possible areas of improvement.

Machine face recognition has traditionally been studied under the assumption of a carefully controlled image acquisition process. By controlling image acquisition, variation due to factors such as pose, lighting, and background can be either largely eliminated or specifically limited to a study over a discrete number of possibilities. Applications of face recognition have had mixed success when deployed in conditions where the assumption of controlled image acquisition no longer holds. This dissertation focuses on this unconstrained face recognition problem, where face images exhibit the same amount of variability that one would encounter in everyday life. We formalize unconstrained face recognition as a binary pair matching problem (verification), and present a data set for benchmarking performance on the unconstrained face verification task. We observe that it is comparatively much easier to obtain many examples of unlabeled face images than face images that have been labeled with identity or other higher level information, such as the position of the eyes and other facial features. We thus focus on improving unconstrained face verification by leveraging the information present in this source of weakly supervised data. We first show how unlabeled face images can be used to perform unsupervised face alignment, thereby reducing variability in pose and improving verification accuracy. Next, we demonstrate how deep learning can be used to perform unsupervised feature discovery, providing additional image representations that can be combined with representations from standard hand-crafted image descriptors, to further improve recognition performance. Finally, we combine unsupervised feature learning with joint face alignment, leading to an unsupervised alignment system that achieves gains in recognition performance matching that achieved by supervised alignment.

The mammalian visual system is an incredibly complex computation device, capable of performing the various tasks of seeing: navigation, pattern and object recognition, motor coordination, trajectory extrapolation, among others. Decades of research has shown that experience-dependent plasticity of cortical circuitry underlies the impressive ability to rapidly learn many of these tasks and to adjust as required. One particular thread of investigation has focused on unsupervised learning, wherein changes to the visual environment lead to corresponding changes in cortical circuits. The most prominent example of unsupervised learning is ocular dominance plasticity, caused by visual deprivation to one eye and leading to a dramatic re-wiring of cortex. Other examples tend to make more subtle changes to the visual environment through passive exposure to novel visual stimuli. Here, we use one such unsupervised paradigm, sequence learning, to study experience-dependent plasticity in the mouse visual system. Through a combination of theory and experiment, we argue that the mammalian visual system is an unsupervised learning device. Beginning with a mathematical exploration of unsupervised learning in biology, engineering, and machine learning, we seek a more precise expression of our fundamental hypothesis. We draw connections between information theory, efficient coding, and common unsupervised learning algorithms such as Hebbian plasticity and principal component analysis. Efficient coding suggests a simple rule for transmitting information in the nervous system: use more spikes to encode unexpected information, and fewer spikes to encode expected information. Therefore, expectation violations ought to produce prediction errors, or brief periods of heightened firing when an unexpected event occurs. Meanwhile, modern unsupervised learning algorithms show how such expectations can be learned. Next, we review data from decades of visual neuroscience research, highlighting the computational principles and synaptic plasticity processes that support biological learning and seeing. By tracking the flow of visual information from the retina to thalamus and primary visual cortex, we discuss how the principle of efficient coding is evident in neural activity. One common example is predictive coding in the retina, where ganglion cells with canonical center-surround receptive fields compute a prediction error, sending spikes to the central nervous system only in response to locally-unpredictable visual stimuli. This behavior can be learned through simple Hebbian plasticity mechanisms. Similar models explain much of the activity of neurons in primary visual cortex, but we also discuss ways in which the theory fails to capture the rich biological complexity. Finally, we present novel experimental results from physiological investigations of the mouse primary visual cortex. We trained mice by passively exposing them to complex spatiotemporal patterns of light: rapidly-flashed sequences of images. We find evidence that visual cortex learns these sequences in a manner consistent with efficient coding, such that unexpected stimuli tend to elicit more firing than expected ones. Overall, we observe dramatic changes in evoked neural activity across days of passive exposure. Neural responses to the first, unexpected sequence element increase with days of training while responses at other, expected time points either decrease or stay the same. Furthermore, substituting an unexpected element for an expected one or omitting an expected element both cause brief bursts of increased firing. Our results therefore provide evidence for unsupervised learning and efficient coding in the mouse visual system, especially because unexpected events drive prediction errors. Overall, our analysis suggests novel experiments, which could be performed in the near future, and provides a useful framework to understand visual perception and learning.

This work aims at developing a novel machine learning method to investigate heterogeneity in neurodevelopmental disorders, with a focus on autism spectrum conditions (ASCs). In ASCs, heterogeneity is shown at several levels of analysis, e.g., genetic, behavioral, throughout developmental trajectories, which hinders the development of effective treatments and the identification of biological pathways involved in gene-cognition-behavior links. ASC diagnosis comes from behavioral observations, which determine the cohort composition of studies in every scientific field (e.g., psychology, neuroscience, genetics). Thus, uncovering behavioral subtypes can provide stratified ASC cohorts that are more representative of the true population. Ideally, behavioral stratification can (1) help to revise and shorten the diagnostic process highlighting the characteristics that best identify heterogeneity; (2) help to develop personalized treatments based on their effectiveness for subgroups of subjects; (3) investigate how the longitudinal course of the condition might differ (e.g., divergent/convergent developmental trajectories); (4) contribute to the identification of genetic variants that may be overlooked in case-control studies; and (5) identify possible disrupted neuronal activity in the brain (e.g., excitatory/inhibitory mechanisms). The characterization of the temporal aspects of heterogeneous manifestations based on their multi-dimensional features is thus the key to identify the etiology of such disorders and establish personalized treatments. Features include trajectories described by a multi-modal combination of electronic health records (EHRs), cognitive functioning and adaptive behavior indicators. This thesis contributes in particular to a data-driven discovery of clinical and behavioral trajectories of individuals with complex disorders and ASCs. Machine learning techniques, such as deep learning and word embedding, that proved successful for e.g., natural language processing and image classification, are gaining ground in healthcare research for precision medicine. Here, we leverage these methods to investigate the feasibility of learning data-driven pathways that have been difficult to identify in the clinical practice to help disentangle the complexity of conditions whose etiology is still unknown. In Chapter 1, we present a new computational method, based on deep learning, to stratify patients with complex disorders; we demonstrate the method on multiple myeloma, Alzheimer’s disease, and Parkinson’s disease, among others. We use clinical records from a heterogeneous patient cohort (i.e., multiple disease dataset) of 1.6M temporally-ordered EHR sequences from the Mount Sinai health system’s data warehouse to learn unsupervised patient representations. These representations are then leveraged to identify subgroups within complex condition cohorts via hierarchical clustering. We investigate the enrichment of terms that code for comorbidities, medications, laboratory tests and procedures, to clinically validate our results. A data analysis protocol is developed in Chapter 2 that produces behavioral embeddings from observational measurements to represent subjects with ASCs in a latent space able to capture multiple levels of assessment (i.e., multiple tests) and the temporal pattern of behavioral-cognitive profiles. The computational framework includes clustering algorithms and state-of-the-art word and text representation methods originally developed for natural language processing. The aim is to detect subgroups within ASC cohorts towards the identification of possible subtypes based on behavioral, cognitive, and functioning aspects. The protocol is applied to ASC behavioral data of 204 children and adolescents referred to the Laboratory of Observation Diagnosis and Education (ODFLab) at the University of Trento. In Chapter 3 we develop a case study for ASCs. From the learned representations of Chapter 1, we select 1,439 individuals with ASCs and investigate whether such representations generalize well to any disorder. Specifically, we identify three subgroups within individuals with ASCs that are further clinically validated to detect clinical profiles based on different term enrichment that can inform comorbidities, therapeutic treatments, medication side effects, and screening policies. This work has been developed in partnership with ODFLab (University of Trento) and the Predictive Models for Biomedicine and Environment unit at FBK. The study reported in Chapter 1 has been conducted at the Institute for Next Generation Healthcare, Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai (NY).

Internet services, such as online shopping and chat apps, have been spreading significantly in recent years, generating substantial amounts of data. These data are precious for machine learning and consist of connections between different entities, such as users and items. These connections contain important information essential for ML models to exploit, and the need to extract this information from graphs gives rise to Graph Representation Learning. By training on these data using Graph Representation Learning methods, hidden information can be obtained, and services can be improved. Initially, the models used for Graph Representation Learning were unsupervised, such as the Deepwalk and Node2vec. These models originated from the field of Natural Language Processing. These models are easy to apply, but their performance is not satisfactory. On the other hand, while supervised models like GNN and GCN have better performance than unsupervised models, they require a huge effort to label the data and finetune the model. Nowadays, the datasets have become larger and more complex, which makes the burden heavier for applying these supervised models. A recent breakthrough in the field of Natural Language Processing may solve the problem. In the paper ‘Attention is all you need’, the authors introduce the Transformer model, which shows excellent performance in NLP. Considering that the field of NLP has many things in common with the GRL and the first supervised models all originated from NLP, it is reasonable to guess whether we can take advantage of the Transformer in improving the performance of the unsupervised model in GRL. Generating embedding for nodes in the graph is one of the significant tasks of GRL. In this thesis, the performance of the Transformer model on generating embedding is tested. Three popular datasets (Cora, Citeseer, Pubmed) are used in training, and the embedding quality is measured through node classification with a linear classification algorithm. Another part of the thesis is to finetune the model to determine the effect of model parameters on embedding accuracy. In this part, comparison experiments are conducted on the dimensions, the number of layers, the sample size, and other parameters. The experiments show that the Transformer model performs better in generating embedding than the original methods, such as the Deepwalk. Compared to supervised methods, it requires less finetuning and less training time. The characteristic of the Transformer model revealed from the experiments shows that it is a good alternative to the baseline model for embedding generation. Improvement may be made on the prepossessing and loss function of the model to get higher performance. / Internettjänster, som onlineshopping och chattappar, har spridits avsevärt de senaste åren och genererat betydande mängder data. Dessa data är värdefulla för maskininlärning och består av kopplingar mellan olika enheter, såsom användare och objekt. Dessa kopplingar innehåller viktig information som är väsentlig för ML-modeller att utnyttja, och behovet av att extrahera denna information från grafer ger upphov till Graph Representation Learning. Genom att träna på dessa data med hjälp av Graph Representation Learning-metoder kan dold information erhållas och tjänster kan förbättras. Till en början var modellerna som användes för Graph Representation Learning oövervakade, såsom Deepwalk och Node2vec. Dessa modeller härstammar från området Natural Language Processing. Dessa modeller är lätta att applicera, men deras prestanda är inte tillfredsställande. Å andra sidan, medan övervakade modeller som GNN och GCN har bättre prestanda än oövervakade modeller, kräver de en enorm ansträngning för att märka data och finjustera modellen. Numera har datamängderna blivit större och mer komplexa, vilket gör bördan tyngre för att tillämpa dessa övervakade modeller. Ett nyligen genomfört genombrott inom området Natural Language Processing kan lösa problemet. I tidningen ‘Attention is all you need’ introducerar författarna Transformer-modellen, som visar utmärkta prestanda i NLP. Med tanke på att området NLP har många saker gemensamt med GRL och att de första övervakade modellerna alla härstammar från NLP, är det rimligt att gissa om vi kan dra fördel av Transformatorn för att förbättra prestandan för den oövervakade modellen i GRL. Att generera inbäddning för noder i grafen är en av GRL:s viktiga uppgifter. I detta examensarbete testas transformatormodellens prestanda för att generera inbäddning. Tre populära datamängder (Cora, Citeseer, Pubmed) används i utbildningen, och inbäddningskvaliteten mäts genom nodklassificering med en linjär klassificeringsalgoritm. En annan del av avhandlingen är att finjustera modellen för att bestämma effekten av modellparametrar på inbäddningsnoggrannheten. I den här delen utförs jämförelseexperiment på dimensionerna, antalet lager, provstorleken och andra parametrar. Experimenten visar att Transformer-modellen presterar bättre när det gäller att generera inbäddning än de ursprungliga metoderna, såsom Deep-walk. Jämfört med övervakade metoder kräver det mindre finjustering och mindre träningstid. Den egenskap hos transformatormodellen som avslöjades från experimenten visar att den är ett bra alternativ till baslinjemodellen för inbäddningsgenerering. Förbättringar kan göras av modellens preposseing- och förlustfunktion för att få högre prestanda.

No description available.

No description available.