Convolutional Neural Networks (CNNs) have been proved to deliver great results in the area of computer vision, however, one fundamental bottleneck with CNNs is the fact that it is heavily dependant on the ground truth, that is, labeled training data. A labeled dataset is a group of samples that have been tagged with one or more labels. In this degree project, we mitigate the data greedy behavior of CNNs by applying deep active learning with various kinds of sampling strategies. The main focus will be on the sampling strategies random sampling, least confidence sampling, margin sampling, entropy sampling, and K- means sampling. We choose to study the random sampling strategy since it will work as a baseline to the other sampling strategies. Moreover, the least confidence sampling, margin sampling, and entropy sampling strategies are uncertainty based sampling strategies, hence, it is interesting to study how they perform in comparison with the geometrical based K- means sampling strategy. These sampling strategies will help to find the most informative/representative samples amongst all unlabeled samples, thus, allowing us to label fewer samples. Furthermore, the benchmark datasets MNIST and CIFAR10 will be used to verify the performance of the various sampling strategies. The performance will be measured in terms of accuracy and less data needed. Lastly, we concluded that by using least confidence sampling and margin sampling we reduced the number of labeled samples by 79.25% in comparison with the random sampling strategy for the MNIST dataset. Moreover, by using entropy sampling we reduced the number of labeled samples by 67.92% for the CIFAR10 dataset. / Faltningsnätverk har visat sig leverera bra resultat inom området datorseende, men en fundamental flaskhals med Faltningsnätverk är det faktum att den är starkt beroende av klassificerade datapunkter. I det här examensarbetet hanterar vi Faltningsnätverkens giriga beteende av klassificerade datapunkter genom att använda deep active learning med olika typer av urvalsstrategier. Huvudfokus kommer ligga på urvalsstrategierna slumpmässigt urval, minst tillförlitlig urval, marginal baserad urval, entropi baserad urval och K- means urval. Vi väljer att studera den slumpmässiga urvalsstrategin eftersom att den kommer användas för att mäta prestandan hos de andra urvalsstrategierna. Dessutom valde vi urvalsstrategierna minst tillförlitlig urval, marginal baserad urval, entropi baserad urval eftersom att dessa är osäkerhetsbaserade strategier som är intressanta att jämföra med den geometribaserade strategin K- means. Dessa urvalsstrategier hjälper till att hitta de mest informativa/representativa datapunkter bland alla oklassificerade datapunkter, vilket gör att vi behöver klassificera färre datapunkter. Vidare kommer standard dastaseten MNIST och CIFAR10 att användas för att verifiera prestandan för de olika urvalsstrategierna. Slutligen drog vi slutsatsen att genom att använda minst tillförlitlig urval och marginal baserad urval minskade vi mängden klassificerade datapunkter med 79, 25%, i jämförelse med den slumpmässiga urvalsstrategin, för MNIST- datasetet. Dessutom minskade vi mängden klassificerade datapunkter med 67, 92% med hjälp av entropi baserad urval för CIFAR10datasetet.

Machine learning for image classification is a hot topic and it is increasing in popularity. Therefore the aim of this study is to provide a better understanding of convolutional neural network hyperparameters by comparing the test accuracy of convolutional neural network models with different hyperparameter value configurations. The focus of this study is to see whether there is an influence in the learning process depending on which hyperparameter values were used. For conducting the experiments convolutional neural network models were developed using the programming language Python utilizing the library Keras. The dataset used for this study iscifar-10, it includes 60000 colour images of 10 categories ranging from man-made objects to different animal species. Grid search is used for instantiating models with varying learning rate and momentum, width and depth values. Learning rate is only tested combined with momentum and width is only tested combined with depth. Activation functions, convolutional layers and batch size are tested individually. Grid search is compared against Bayesian optimization to see which technique will find the most optimized learning rate and momentum values. Results illustrate that the impact different hyperparameters have on the overall test accuracy varies. Learning rate and momentum affects the test accuracy greatly, however suboptimal values for learning rate and momentum can decrease the test accuracy severely. Activation function, width and depth, convolutional layer and batch size have a lesser impact on test accuracy. Regarding Bayesian optimization compared to grid search, results show that Bayesian optimization will not necessarily find more optimal hyperparameter values.

För att förutsäga snö är en av de viktigaste redskapen en snömodell som beskriver hur snö ackumuleras och avsmälter. En viktig aspekt i snömodellering är variationmed höjden. Höjden påverkar temperatur och nederbörd och därigenom också mönstret för avsmältning och ackumulering.En grad-dag snömodell över området anslutande till Kultsjöns avrinningsområde utfördes med hänsyn till höjdfördelningen. Modellens snötäcke kalibrerades med hjälp av klassificerade satellitfoton över området under perioden mars till juni 2014. Jämförelsen gjordes med hjälp av Cohens Kappa.Resultatet av simuleringen påvisade en påtaglig överrensstämmelse mellan modellen och den observerade data. De simulerade värdena för snödjup jämfördes med observerade data för att utföra en enkel validering. Igen erhölls till stor del överrensstämmelse.Det finns säkert ett behov av tillägg till modellen som tar hänsyn till strålning och vind, då båda dessa faktorer uteblev i modellen. / To predict snow, one of the most important tools is a snow model that describes how snow accumulates and melts. An important aspect in snow modeling is variation with elevation. Elevation influences temperature and precipitation, and therefore also the patterns of snow melt and accumulation.A degree-day snow model over the area around Kultsjön’s catchment area was made with respect to elevation distribution. The modeled snow cover was calibrated using classified satellite photo over the area during the period March to June 2014. The comparison was done using Cohen’s Kappa.The results of the simulation show a large portion of agreement between the model and observed data. The simulated values for snow depth were then compared to the observed data to perform a basic validation. Again there was a large portion of agreement.There is certainly a need for supplementary adjustments to the model that take into account radiation and wind, as both factors were left out of the model.

No description available.

<p>Domain adaptation is one of the hottest directions in solving annotation insufficiency problem of deep learning. General domain adaptation is not consistent with the practical scenarios in the industry. In this thesis, we focus on two concerns as below.</p> <p>  </p> <p>  First is that labeled data are generally collected from multiple domains. In other words, multi-source adaptation is a more common situation. Simply extending these single-source approaches to the multi-source cases could cause sub-optimal inference, so specialized multi-source adaptation methods are essential. The main challenge in the multi-source scenario is a more complex divergence situation. Not only the divergence between target and each source plays a role, but the divergences among distinct sources matter as well. However, the significance of maintaining consistency among multiple sources didn't gain enough attention in previous work. In this thesis, we propose an Enhanced Consistency Multi-Source Adaptation (EC-MSA) framework to address it from three perspectives. First, we mitigate feature-level discrepancy by cross-domain conditional alignment, narrowing the divergence between each source and target domain class-wisely. Second, we enhance multi-source consistency via dual mix-up, diminishing the disagreements among different sources. Third, we deploy a target distilling mechanism to handle the uncertainty of target prediction, aiming to provide high-quality pseudo-labeled target samples to benefit the previous two aspects. Extensive experiments are conducted on several common benchmark datasets and demonstrate that our model outperforms the state-of-the-art methods.</p> <p>  </p> <p>  Second is that data privacy and security is necessary in practice. That is, we hope to keep the raw data stored locally while can still obtain a satisfied model. In such a case, the risk of data leakage greatly decreases. Therefore, it is natural for us to combine the federated learning paradigm with domain adaptation. Under the source-private setting, the main challenge for us is to expose information from the source domain to the target domain while make sure that the communication process is safe enough. In this thesis, we propose a method named Fourier Transform-Assisted Federated Domain Adaptation (FTA-FDA) to alleviate the difficulties in two ways. We apply Fast Fourier Transform to the raw data and transfer only the amplitude spectra during the communication. Then frequency space interpolations between these two domains are conducted, minimizing the discrepancies while ensuring the contact of them and keeping raw data safe. What's more, we make prototype alignments by using the model weights together with target features, trying to reduce the discrepancy in the class level. Experiments on Office-31 demonstrate the effectiveness and competitiveness of our approach, and further analyses prove that our algorithm can help protect privacy and security.</p>

Data augmentation is a technique that acquires more training data by augmenting available samples, where the training data is used to fit model parameters. Data augmentation is utilized due to a shortage of training data in certain domains and to reduce overfitting. Augmenting a training dataset for image classification with a Generative Adversarial Network (GAN) has been shown to increase classification accuracy. This report investigates if transfer learning within a GAN can further increase classification accuracy when utilizing the augmented training dataset. The method section describes a specific GAN architecture for the experiments that includes a label condition. When using transfer learning within the specific GAN architecture, a statistical analysis shows a statistically significant increase in classification accuracy for a classification problem with the EMNIST dataset, which consists of images of handwritten alphanumeric characters. In the discussion section, the authors analyze the results and motivates other use cases for the proposed GAN architecture. / Datautökning är en metod som skapar mer träningsdata genom att utöka befintlig träningsdata, där träningsdatan används för att anpassa modellers parametrar. Datautökning används på grund av en brist på träningsdata inom vissa områden samt för att minska overfitting. Att utöka ett träningsdataset för att genomföra bildklassificering med ett generativt adversarialt nätverk (GAN) har visats kunna öka precisionen av klassificering av bilder. Denna rapport undersöker om transferlärande inom en GAN kan vidare öka klassificeringsprecisionen när ett utökat träningsdataset används. Metoden beskriver en specific GANarkitektur som innehåller ett etikettvillkor. När transferlärande används inom den utvalda GAN-arkitekturen visar en statistisk analys en statistiskt säkerställd ökning av klassificeringsprecisionen för ett klassificeringsproblem med EMNIST datasetet, som innehåller bilder på handskrivna bokstäver och siffror. I diskussionen diskuteras orsakerna bakom resultaten och fler användningsområden nämns.

The advances in deep neural networks in recent years have opened up the possibility of using image classification as a valuable tool in various areas, such as medical diagnosis from x-ray images. However, training deep neural networks requires large amounts of annotated data which has to be labelled manually, by a person. This process always involves a risk of data getting the wrong label, either by mistake or ill will, and training a machine learning model on mislabelled images has a negative impact on accuracy. Studies have shown that deep neural networks are so powerful at memorization that if they train on mislabelled data, they will eventually overfit this data, meaning learning a data representation that does not fully mirror real data. It is therefore vital to filter out these images. Area under the margin is a method that filters out mislabelled images by observing the changes in a network’s predictions during training. This method does however not take into consideration the overconfidence in deep neural networks and the uncertainty of a model can give indications of mislabelled images during training. Calibrating the confidence can be done through label smoothing and this thesis aims to investigate if the performance of Area under the margin can be improved when combined with different smoothing techniques. The goal is to develop a better insight into how different types of label noise affects models in terms of confidence, accuracy and the impact it has depending on the dataset itself. Three different label smoothing techniques will be applied to evaluate how well they can mitigate overconfidence, prevent the model from memorizing the mislabelled samples and if this can improve the filtering process for the Area under the margin method. Results show when training on data with noise present, adding label smoothing improves accuracy, an indication of noise robustness. Label noise is seen to decrease confidence in the model and at the same time reduce the calibration. Adding label smoothing prevents this and allows the model to be more robust as the noise rate increases. In the filtering process, label smoothing was seen to prevent correctly labelled samples to be filtered and received a better accuracy at identifying the noise. This did not improve the classification results on the filtered data, indicating that it is more important to filter out as many mislabelled samples as possible even if this means filtering out correctly labelled images as well. The label smoothing methods used in this work was set up to preserve calibration, a future topic of research could be to adjust the hyperparameters to increase confidence instead, focusing on removing as much noise as possible. / De senaste årens framsteg inom djupa neurala nätverk har öppnat för möjligheten att använda bildklassificering som ett värdefullt verktyg inom olika områden, såsom medicinsk diagnos från röntgenbilder. Men att träna djupa neurala nätverk kräver stora mängder annoterad data som måste märkas antingen av människor eller datorer. Denna process involverar alltid med en risk för att data får fel etikett, antingen av misstag eller av uppsåt och att träna en maskininlärningsmodell på felmärkta bilder har negativ inverkan på resultatet. Studier har visat att djupa neurala nätverk är så kraftfulla att memorera att om de tränar på felmärkta data, kommer de så småningom att överanpassa dessa data, vilket betyder att de kommer att lära sig en representation som inte helt speglar verklig data. Det är därför viktigt att filtrera bort dessa bilder. Area under marginalen är en metod som filtrerar bort felmärkta bilder genom att observera förändringarna i ett nätverks beteende under träning. Denna metod tar dock inte hänsyn till översäkerhet i djupa neurala nätverk och osäkerheten i en modell kan ge indikationer på felmärkta bilder under träning. Kalibrering av förtroendet kan göras genom etikettutjämning och denna uppsats syftar till att undersöka om prestandan för Area under marginalen kan förbättras i kombination med olika tekniker för etikettutjämning. Målet är att utveckla en bättre insikt i hur olika typer av brusiga etiketter påverkar modeller när det gäller tillförlitlighet, noggrannhet och den påverkan det har beroende på själva datasetet. Tre olika tekniker för etikettutjämning kommer att tillämpas för att utvärdera hur väl de kan mildra översäkerheten, förhindra modellen från att memorera de felmärkta bilderna och om detta kan förbättra filtreringsprocessen för Area under marginalen-metoden. Resultaten visar att när man tränar på data innehållande felmärkt data, förbättrar etikettutjämning noggrannheten vilket indikerar på robusthet mot felmärkning. Felmärkning tycks minska säkerheten hos modellen och samtidigt minska kalibreringen. Att lägga till etikettutjämning förhindrar detta och gör att modellen blir mer robust när mängden brusiga etiketter ökar. I filtreringsprocessen sågs att etikettutjämning förhindrar att korrekt märkt data filtreras bort och fick en bättre noggrannhet vid identifiering av bruset. Detta förbättrade dock inte klassificeringsresultaten på den filtrerade datan, vilket indikerar att det är viktigare att filtrera bort så mycket felmärkta prover som möjligt även om detta innebär att filtrera bort korrekt märkta bilder. Metoderna för etikettutjämning som används i detta arbete sattes upp för att bevara kalibreringen, ett framtida forskningsämne kan vara att justera hyperparametrarna för att istället öka förtroendet, med fokus på att ta bort så mycket felmärkta etiketter som möjligt.

<p>Statistical modeling aims at constructing a mathematical model for an existing data set. As a comprehensive concept, statistical modeling leads to a wide range of interesting problems. Modern parametric models, such as deepnets, have achieved remarkable success in quite a few application areas with massive data. Although being powerful in practice, many fitted over-parameterized models potentially suffer from losing good statistical properties. For this reason, a new framework named the Auto-modeling (AM) framework is proposed. Philosophically, the mindset is to fit models to future observations rather than the observed sample. Technically, choosing an imputation model for generating future observations, we fit models to future observations via optimizing an approximation to the desired expected loss function based on its sample counterpart and what we call an adaptive {\it duality function}.</p> <p><br></p> <p>The first part of the dissertation (Chapter 2 to 7) focuses on the new philosophical perspective of the method, as well as the details of the main framework. Technical details, including essential theoretical properties of the method are also investigated. We also demonstrate the superior performance of the proposed method via three applications: Many-normal-means problem, $n < p$ linear regression and image classification.</p> <p><br></p> <p>The second part of the dissertation (Chapter 8) focuses on the application of the AM framework to the construction of linear regression models. Our primary objective is to shed light on the stability issue associated with the commonly used data-driven model selection methods such as cross-validation (CV). Furthermore, we highlight the philosophical distinctions between CV and AM. Theoretical properties and numerical examples presented in the study demonstrate the potential and promise of AM-based linear model selection. Additionally, we have devised a conformal prediction method specifically tailored for quantifying the uncertainty of AM predictions in the context of linear regression.</p>

[pt] Modelos baseados em redes neurais profundas como as Redes Neurais Convolucionais proporcionaram avanços significativos em diversas áreas da computação. No entanto, essa tecnologia é ainda pouco aplicada à predição de qualidade sísmica, que é uma atividade relevante para exploração de hidrocarbonetos. Ser capaz de, rapidamente, classificar o ruído presente em aquisições de dados sísmicos permite aceitar ou rejeitar essas aquisições de forma eficiente, o que além de economizar recursos também melhora a interpretabilidade dos dados. Neste trabalho apresenta-se um dataset criado a partir de 6.918 aquisições manualmente classificadas pela percepção de especialistas e pesquisadores, que serviu de base para o treinamento, validação e testes de um classificador, também proposto neste trabalho, baseado em uma rede neural convolucional. Em resultados empíricos, observou-se-se um F1 Score de 95,58 porcento em uma validação cruzada de 10 folds e 93,56 porcento em um conjunto de holdout de teste. / [en] Deep Learning based models, such as Convolutional Neural Networks (CNNs), have led to significant advances in several areas of computing applications. Nevertheless, this technology is still rarely applied to seismic quality prediction, which is a relevant task in hydrocarbon exploration. Being able to promptly classify noise in common shot gather(CSG) acquisitions of seismic data allows the acceptance or rejection of those aquisitions, not only saving resources but also increasing the interpretability of data. In this work, we introduce a real-world classification dataset based on 6.918 common shot gather, manually labeled by perception of specialists and researches. We use it to train a CNN classification model for seismic shot-gathers quality prediction. In our empirical evaluation, we observed an F1 Score of 95,58 percent in 10 fold cross-validation and 93,56 percent in a Holdout Test.

Image classification in Machine Learning encompasses the task of identification of objects in an image. The technique has applications in various areas such as e-commerce, social media and security surveillance. In this report the author explores the impact of using t-Distributed Stochastic Neighbor Embedding (t-SNE) on data as a preprocessing step when classifying multiple classes of clothing with a state-of-the-art Deep Convolutional Neural Network (DCNN). The t-SNE algorithm uses dimensionality reduction and groups similar objects close to each other in three-dimensional space. Extracting this information in the form of a positional coordinate gives us a new parameter which could help with the classification process since the features it uses can be different from that of the DCNN. Therefore, three slightly different DCNN models receives different input and are compared. The first benchmark model only receives pixel values, the second and third receive pixel values together with the positional coordinates from the t-SNE preprocessing for each data point, but with different hyperparameter values in the preprocessing step. The Fashion-MNIST dataset used contains 10 different clothing classes which are normalized and gray-scaled for easeof-use. The dataset contains 70.000 images in total. Results show minimum change in classification accuracy in the case of using a low-density map with higher learning rate as the data size increases, while a more dense map and lower learning rate performs a significant increase in accuracy of 4.4% when using a small data set. This is evidence for the fact that the method can be used to boost results when data is limited. / Bildklassificering i maskinlärning innefattar uppgiften att identifiera objekt i en bild. Tekniken har applikationer inom olika områden så som e-handel, sociala medier och säkerhetsövervakning. I denna rapport undersöker författaren effekten av att användat-Distributed Stochastic Neighbour Embedding (t-SNE) på data som ett förbehandlingssteg vid klassificering av flera klasser av kläder med ett state-of-the-art Deep Convolutio-nal Neural Network (DCNN). t-SNE-algoritmen använder dimensioneringsreduktion och grupperar liknande objekt nära varandra i tredimensionellt utrymme. Att extrahera denna information i form av en positionskoordinat ger oss en ny parameter som kan hjälpa till med klassificeringsprocessen eftersom funktionerna som den använder kan skilja sig från DCNN-modelen. Tre olika DCNN-modeller får olika in-data och jämförs därefter. Den första referensmodellen mottar endast pixelvärden, det andra och det tredje motar pixelvärden tillsammans med positionskoordinaterna från t-SNE-förbehandlingen för varje datapunkt men med olika hyperparametervärden i förbehandlingssteget. I studien används Fashion-MNIST datasetet som innehåller 10 olika klädklasser som är normaliserade och gråskalade för enkel användning. Datasetet innehåller totalt 70.000 bilder. Resultaten visar minst förändring i klassificeringsnoggrannheten vid användning av en låg densitets karta med högre inlärningsgrad allt eftersom datastorleken ökar, medan en mer tät karta och lägre inlärningsgrad uppnår en signifikant ökad noggrannhet på 4.4% när man använder en liten datamängd. Detta är bevis på att metoden kan användas för att öka klassificeringsresultaten när datamängden är begränsad.