L’apprentissage machine est un vaste domaine où l’on cherche à apprendre les paramètres de modèles à partir de données concrètes. Ce sera pour effectuer des tâches demandant des aptitudes attribuées à l’intelligence humaine, comme la capacité à traiter des don- nées de haute dimensionnalité présentant beaucoup de variations. Les réseaux de neu- rones artificiels sont un exemple de tels modèles. Dans certains réseaux de neurones dits profonds, des concepts "abstraits" sont appris automatiquement. Les travaux présentés ici prennent leur inspiration de réseaux de neurones profonds, de réseaux récurrents et de neuroscience du système visuel. Nos tâches de test sont la classification et le débruitement d’images quasi binaires. On permettra une rétroac- tion où des représentations de haut niveau (plus "abstraites") influencent des représentations à bas niveau. Cette influence s’effectuera au cours de ce qu’on nomme relaxation, des itérations où les différents niveaux (ou couches) du modèle s’interinfluencent. Nous présentons deux familles d’architectures, l’une, l’architecture complètement connectée, pouvant en principe traiter des données générales et une autre, l’architecture convolutionnelle, plus spécifiquement adaptée aux images. Dans tous les cas, les données utilisées sont des images, principalement des images de chiffres manuscrits. Dans un type d’expérience, nous cherchons à reconstruire des données qui ont été corrompues. On a pu y observer le phénomène d’influence décrit précédemment en comparant le résultat avec et sans la relaxation. On note aussi certains gains numériques et visuels en terme de performance de reconstruction en ajoutant l’influence des couches supérieures. Dans un autre type de tâche, la classification, peu de gains ont été observés. On a tout de même pu constater que dans certains cas la relaxation aiderait à apprendre des représentations utiles pour classifier des images corrompues. L’architecture convolutionnelle développée, plus incertaine au départ, permet malgré tout d’obtenir des reconstructions numériquement et visuellement semblables à celles obtenues avec l’autre architecture, même si sa connectivité est contrainte. / Machine learning is a vast field where we seek to learn parameters for models from concrete data. The goal will be to execute various tasks requiring abilities normally associated more with human intelligence than with a computer program, such as the ability to process high dimensional data containing a lot of variations. Artificial neural networks are a large class of such models. In some neural networks said to be deep, we can observe that high level (or "abstract") concepts are automatically learned. The work we present here takes its inspiration from deep neural networks, from recurrent networks and also from neuroscience of the visual system. Our test tasks are classification and denoising for near binary images. We aim to take advantage of a feedback mechanism through which high-level representations, that is to say relatively abstract concepts, can influence lower-level representations. This influence will happen during what we call relaxation, which is iterations where the different levels (or layers) of the model can influence each other. We will present two families of architectures based on this mechanism. One, the fully connected architecture, can in principle accept generic data. The other, the convolutional one, is specifically made for images. Both were trained on images, though, and mostly images of written characters. In one type of experiment, we want to reconstruct data that has been corrupted. In these tasks, we have observed the feedback influence phenomenon previously described by comparing the results we obtained with and without relaxation. We also note some numerical and visual improvement in terms of reconstruction performance when we add upper layers’ influence. In another type of task, classification, little gain has been noted. Still, in one setting where we tried to classify noisy data with a representation trained without prior class information, relaxation did seem to improve results significantly. The convolutional architecture, a bit more risky at first, was shown to produce numerical and visual results in reconstruction that are near those obtained with the fully connected version, even though the connectivity is much more constrained.

Recent improvements in deep learning architectures, combined with the strength of modern computing hardware such as graphics processing units, has lead to significant results in the field of image analysis. In this thesis work, locally connected architectures are employed to reduce noise in flash X-ray diffraction images. The layers in these architectures use convolutional kernels, but without shared weights. This combines the benefits of lower model memory footprint in convolutional networks with the higher model capacity of fully connected networks. Since the camera used to capture the diffraction images has pixelwise unique characteristics, and thus lacks equivariance, this compromise can be beneficial. The background images of this thesis work were generated with an active laser but without injected samples. Artificial diffraction patterns were then added to these background images allowing for training U-Net architectures to separate them. Architecture A achieved a performance of 0.187 on the test set, roughly translating to 35 fewer photon errors than a model similar to state of the art. After smoothing the photon errors this performance increased to 0.285, since the U-Net architectures managed to remove flares where state of the art could not. This could be taken as a proof of concept that locally connected networks are able to separate diffraction from background in flash X-Ray imaging.

No description available.

<div><div><div><p>Many complex processes can be viewed as sequential events on a network. In this thesis, we study the interplay between a network and the event sequences on it. We first focus on predicting events on a known network. Examples of such include: modeling retweet cascades, forecasting earthquakes, and tracing the source of a pandemic. In specific, given the network structure, we solve two types of problems - (1) forecasting future events based on the historical events, and (2) identifying the initial event(s) based on some later observations of the dynamics. The inverse problem of inferring the unknown network topology or links, based on the events, is also of great important. Examples along this line include: constructing influence networks among Twitter users from their tweets, soliciting new members to join an event based on their participation history, and recommending positions for job seekers according to their work experience. Following this direction, we study two types of problems - (1) recovering influence networks, and (2) predicting links between a node and a group of nodes, from event sequences.</p></div></div></div>

The aim of this thesis is to explore clustering algorithms of machine unsupervised learning, which can be used for image database classification by similarity. For chosen clustering algorithms is written up a theoretical basis. For better classification of used database this thesis deals with different methods of image preprocessing. With these methods the features from image are extracted. Next the thesis solves of implementation of preprocessing methods and practical application of clustering algorithms. In practical part is programmed aplication in Python programming language, which classifies the database of images into classes by similarity. The thesis tests all of used methods and at the end of the thesis is processed searches of results.

The thesis addresses the topic of Deep Neural Networks, in particular the methods regar- ding the field of Deep Learning, which is used to initialize the weight and learning process s itself within Deep Neural Networks. The focus is also put to the basic theory of the classical Neural Networks, which is important to comprehensive understanding of the issue. The aim of this work is to determine the optimal set of optional parameters of the algori- thms on various complexity levels of image recognition tasks through experimenting with created application applying Deep Neural Networks. Furthermore, evaluation and analysis of the results and lessons learned from the experimentation with classical and Deep Neural Networks are integrated in the thesis.

Being able to accurately approximate outputs of computationally expensive simulations for arbitrary input parameters, also called missing points estimation, is central in many different areas of research and development with applications ranging from uncertainty propagation to control system design to name a few. This project investigates the potential of kernel transformations and nonlinear autoencoders as methods of improving the accuracy of the proper orthogonal decomposition method combined with regression. The techniques are applied on aerodynamic pressure CFD data around airplane wings in both two- and three-dimensional settings. The novel methods show potential in select situations, but cannot at this stage be generally considered superior. Their performances are similar although the procedure of design and training of a nonlinear autoencoder is less straight forward and more time demanding than using kernel transformations. The results demonstrate the regression bottleneck of the proper orthogonal decomposition method, which partially is improved with the new methods. Future studies should focus on adapting the autoencoder training strategy to the architecture and data as well as improving the regression stage of all methods.

We consider a mechanistic non-linear machine learning approach to learning signals in financial time series data. A modularised and decoupled algorithm framework is established and is proven on daily sampled closing time-series data for JSE equity markets. The input patterns are based on input data vectors of data windows preprocessed into a sequence of daily, weekly and monthly or quarterly sampled feature measurement changes (log feature fluctuations). The data processing is split into a batch processed step where features are learnt using a Stacked AutoEncoder (SAE) via unsupervised learning, and then both batch and online supervised learning are carried out on Feedforward Neural Networks (FNNs) using these features. The FNN output is a point prediction of measured time-series feature fluctuations (log differenced data) in the future (ex-post). Weight initializations for these networks are implemented with restricted Boltzmann machine pretraining, and variance based initializations. The validity of the FNN backtest results are shown under a rigorous assessment of backtest overfitting using both Combinatorially Symmetrical Cross Validation and Probabilistic and Deflated Sharpe Ratios. Results are further used to develop a view on the phenomenology of financial markets and the value of complex historical data under unstable dynamics.

<div>Leaf Area Index (LAI) is an important variable for both for characterizing plant canopy and as an input to many crop models. It is a dimensionless quantity broadly defined as the total one-sided leaf area per unit ground area, and is estimated over agriculture row crops by both direct and indirect methods. Direct methods, which involve destructive sampling, are laborious and time-consuming, while indirect methods such as remote sensing-based approaches have multiple sources of uncertainty. LiDAR (Light Detection and Ranging) remotely sensed data acquired from manned aircraft and UAVs’ have been investigated to estimate LAI based on physical/geometric features such as canopy gap fraction. High-resolution point cloud data acquired with a laser scanner from any platform, including terrestrial laser scanning and mobile mapping systems, contain random noise and outliers. Therefore, outlier detection in LiDAR data is often useful prior to analysis. Applications in agriculture are particularly challenging, as there is typically no prior knowledge of the statistical distribution of points, description of plant complexity, and local point densities, which are crop dependent. This dissertation first explores the effectiveness of using LiDAR data to estimate LAI for row crop plants at multiple times during the growing season from both a wheeled vehicle and an Unmanned Aerial Vehicle (UAV). Linear and nonlinear regression models are investigated for prediction utilizing statistical and plant structure-based features extracted from the LiDAR point cloud data and ground reference obtained from an in-field plant canopy analyzer and leaf area derived from destructive sampling. LAI estimates obtained from support vector regression (SVR) models with a radial basis function (RBF) kernel developed using the wheel-based LiDAR system and UAVs are promising, based on the value of the coefficient of determination (R2) and root mean squared error (RMSE) of the residuals. </div><div>This dissertation also investigates approaches to minimize the impact of outliers on discrete return LiDAR acquired over crops, and specifically for sorghum and maize breeding experiments, by an unmanned aerial vehicle (UAV) and a wheel-based ground platform. Two methods are explored to detect and remove the outliers from the plant datasets. The first is based on surface fitting to noisy point cloud data based on normal and curvature estimation in a local neighborhood. The second utilizes the deep learning framework PointCleanNet. Both methods are applied to individual plants and field-based datasets. To evaluate the method, an F-score and LAI are calculated both before and after outlier removal for both scenarios. Results indicate that the deep learning method for outlier detection is more robust to changes in point densities, level of noise, and shapes. Also, the predicted LAI was improved for the wheel-based vehicle data based on the R2 value and RMSE of residuals. </div><div>The quality of the extracted features depends on the point density and laser penetration of the canopy. Extracting appropriate features is a critical step to have accurate prediction models. Deep learning frameworks are increasingly being used in remote sensing applications. In the last objective of this study, a feature extraction approach is investigated for encoding LiDAR data acquired by UAV platforms multiple times during the growing season over sorghum and maize plant breeding experiments. LAI estimates obtained with these inputs are used to develop support vector regression (SVR) models using plant canopy analyzer data as the ground reference. Results are compared to models based on estimates from physically-based features and evaluated in terms of the coefficient determination (R2). The effects of experimental conditions, including flying height, sensor characteristics, and crop type, are also investigated relative to the estimates of LAI.</div><div><br></div>

Nuclear power plants are inherently complex systems. While the technology has been used to generate electrical power for many decades, process monitoring continuously evolves. There is always room for improvement in terms of maximizing the availability by reducing the risks of problems and errors. In this context, automated monitoring systems have become important tools – not least with the rapid progress being made in the field of data analytics thanks to ever increasing amounts of processing power. There are many different types of models that can be utilized for identifying anomalies. Some rely on physical properties and theoretical relations, while others rely more on the patterns of historical data. In this thesis, a data-driven approach using a hierarchical autoencoder framework has been developed for the purposes of anomaly detection at the Swedish nuclear power plant Forsmark. The model is first trained to recognize normal operating conditions. The trained model then creates reference values and calculates the deviations in relation to real data in order to identify any issues. This proof-of-concept has been evaluated and benchmarked against a currently used hybrid model with more physical modeling properties in order to identify benefits and drawbacks. Generally speaking, the created model has performed in line with expectations. The currently used tool is more flexible in its understanding of different plant states and is likely better at determining root causes thanks to its physical modeling properties. However, the created autoencoder framework does bring other advantages. For instance, it allows for a higher time resolution thanks to its relatively low calculation intensity. Additionally, thanks to its purely data-driven characteristics, it offers great opportunities for future reconfiguration and adaptation with different signal selections.