Deux aspects importants de la connaissance qu'un individu a pu acquérir par ses expériences correspondent à la mémoire sémantique (celle des connaissances explicites, comme par exemple l'apprentissage de concepts et de catégories décrivant les objets du monde) et la mémoire procédurale (connaissances relatives à l'apprentissage de règles ou de la syntaxe). Cette "mémoire syntaxique" se construit à partir de l'expérience et notamment de l'observation de séquences, suites d'objets dont l'organisation séquentielle obéit à des règles syntaxiques. Elle doit pouvoir être utilisée ultérieurement pour générer des séquences valides, c'est-à-dire respectant ces règles. Cette production de séquences valides peut se faire de façon explicite, c'est-à-dire en évoquant les règles sous-jacentes, ou de façon implicite, quand l'apprentissage a permis de capturer le principe d'organisation des séquences sans recours explicite aux règles. Bien que plus rapide, plus robuste et moins couteux en termes de charge cognitive que le raisonnement explicite, le processus implicite a pour inconvénient de ne pas donner accès aux règles et de ce fait, de devenir moins flexible et moins explicable. Ces mécanismes mnésiques s'appliquent aussi à l'expertise métier : la capitalisation des connaissances pour toute entreprise est un enjeu majeur et concerne aussi bien celles explicites que celles implicites. Au début, l'expert réalise un choix pour suivre explicitement les règles du métier. Mais ensuite, à force de répétition, le choix se fait automatiquement, sans évocation explicite des règles sous-jacentes. Ce changement d'encodage des règles chez un individu en général et particulièrement chez un expert métier peut se révéler problématique lorsqu'il faut expliquer ou transmettre ses connaissances. Si les concepts métiers peuvent être formalisés, il en va en général de tout autre façon pour l'expertise. Dans nos travaux, nous avons souhaité nous pencher sur les séquences de composants électriques et notamment la problématique d’extraction des règles cachées dans ces séquences, aspect important de l’extraction de l’expertise métier à partir des schémas techniques. Nous nous plaçons dans le domaine connexionniste, et nous avons en particulier considéré des modèles neuronaux capables de traiter des séquences. Nous avons implémenté deux réseaux de neurones récurrents : le modèle de Elman et un modèle doté d’unités LSTM (Long Short Term Memory). Nous avons évalué ces deux modèles sur différentes grammaires artificielles (grammaire de Reber et ses variations) au niveau de l’apprentissage, de leurs capacités de généralisation de celui-ci et leur gestion de dépendances séquentielles. Finalement, nous avons aussi montré qu’il était possible d’extraire les règles encodées (issues des séquences) dans le réseau récurrent doté de LSTM, sous la forme d’automate. Le domaine électrique est particulièrement pertinent pour cette problématique car il est plus contraint avec une combinatoire plus réduite que la planification de tâches dans des cas plus généraux comme la navigation par exemple, qui pourrait constituer une perspective de ce travail. / There are two important aspects of the knowledge that an individual acquires through experience. One corresponds to the semantic memory (explicit knowledge, such as the learning of concepts and categories describing the objects of the world) and the other, the procedural or syntactic memory (knowledge relating to the learning of rules or syntax). This "syntactic memory" is built from experience and particularly from the observation of sequences of objects whose organization obeys syntactic rules.It must have the capability to aid recognizing as well as generating valid sequences in the future, i.e., sequences respecting the learnt rules. This production of valid sequences can be done either in an explicit way, that is, by evoking the underlying rules, or implicitly, when the learning phase has made it possible to capture the principle of organization of the sequences without explicit recourse to the rules. Although the latter is faster, more robust and less expensive in terms of cognitive load as compared to explicit reasoning, the implicit process has the disadvantage of not giving access to the rules and thus becoming less flexible and less explicable. These mnemonic mechanisms can also be applied to business expertise. The capitalization of information and knowledge in general, for any company is a major issue and concerns both the explicit and implicit knowledge. At first, the expert makes a choice to explicitly follow the rules of the trade. But then, by dint of repetition, the choice is made automatically, without explicit evocation of the underlying rules. This change in encoding rules in an individual in general and particularly in a business expert can be problematic when it is necessary to explain or transmit his or her knowledge. Indeed, if the business concepts can be formalized, it is usually in any other way for the expertise which is more difficult to extract and transmit.In our work, we endeavor to observe sequences of electrical components and in particular the problem of extracting rules hidden in these sequences, which are an important aspect of the extraction of business expertise from technical drawings. We place ourselves in the connectionist domain, and we have particularly considered neuronal models capable of processing sequences. We implemented two recurrent neural networks: the Elman model and a model with LSTM (Long Short Term Memory) units. We have evaluated these two models on different artificial grammars (Reber's grammar and its variations) in terms of learning, their generalization abilities and their management of sequential dependencies. Finally, we have also shown that it is possible to extract the encoded rules (from the sequences) in the recurrent network with LSTM units, in the form of an automaton. The electrical domain is particularly relevant for this problem. It is more constrained with a limited combinatorics than the planning of tasks in general cases like navigation for example, which could constitute a perspective of this work.

The thesis deals with problems of automatic detection of premature ventricular contractions in ECG records. One detection method which uses a convolutional neural network and LSTM units is implemented in the Python language. Cardiac cycles extracted from one-lead ECG were used for detection. F1 score for binary classification (PVC and normal beat) on the test dataset reached 96,41 % and 81,76 % for three-class classification (PVC, normal beat and other arrhythmias). Lastly, the accuracy of the classification is evaluated and discussed, the achieved results for binary classification are comparable to the results of methods described in different papers.

This thesis deals with anomaly detection of log data. Big software systems produce a great amount of log data which are not further processed. There are usually so many logs that it becomes impossible to check every log entry manually. In this thesis we introduce models that minimize primarily count of false positive predictions with expected complexity of data annotation taken into account. The compared models are based on PCA algorithm, N-gram model and recurrent neural networks with LSTM cell. In the thesis we present results of the models on widely used datasets and also on a real dataset provided by HAVIT, s.r.o. 1

System log files are filled with logged events, status codes, and other messages. By analyzing the log files, the systems current state can be determined, and find out if something during its execution went wrong. Log file analysis has been studied for some time now, where recent studies have shown state-of-the-art performance using machine learning techniques. In this thesis, document classification solutions were tested on log files in order to classify regular system runs versus abnormal system runs. To solve this task, supervised and unsupervised learning methods were combined. Doc2Vec was used to extract document features, and Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) based architectures on the classification task. With the use of the machine learning models and preprocessing techniques the tested models yielded an f1-score and accuracy above 95% when classifying log files.

The success of Internet of Things solutions allowed the establishment of new applications such as smart hydroponic agriculture. One typical problem in such an application is the rapid degradation of the deployed sensors. Traditionally, this problem is resolved by frequent manual maintenance, which is considered to be ineffective and may harm the crops in the long run. The main purpose of this thesis was to propose a machine learning approach for automating the detection of sensor fault drifts. In addition, the solution’s operability was investigated in a cloud computing environment in terms of the response time. This thesis proposes a detection algorithm that utilizes RNN in predicting sensor drifts from time-series data streams. The detection algorithm was later named; Predictive Sliding Detection Window (PSDW) and consisted of both forecasting and classification models. Three different RNN algorithms, i.e., LSTM, CNN-LSTM, and GRU, were designed to predict sensor drifts using forecasting and classification techniques. The algorithms were compared against each other in terms of relevant accuracy metrics for forecasting and classification. The operability of the solution was investigated by developing a web server that hosted the PSDW algorithm on an AWS computing instance. The resulting forecasting and classification algorithms were able to make reasonably accurate predictions for this particular scenario. More specifically, the forecasting algorithms acquired relatively low RMSE values as ~0.6, while the classification algorithms obtained an average F1-score and accuracy of ~80% but with a high standard deviation. However, the response time was ~5700% slower during the simulation of the HTTP requests. The obtained results suggest the need for future investigations to improve the accuracy of the models and experiment with other computing paradigms for more reliable deployments.

In this thesis, we analyse the sentiment in equity research reports written by analysts at Skandinaviska Enskilda Banken (SEB). We provide a description of established statistical and machine learning methods for classifying the sentiment in text documents as positive or negative. Specifically, a form of recurrent neural network known as long short-term memory (LSTM) is of interest. We investigate two different labelling regimes for generating training data from the reports. Benchmark classification accuracies are obtained using logistic regression models. Finally, two different word embedding models and bidirectional LSTMs of varying network size are implemented and compared to the benchmark results. We find that the logistic regression works well for one of the labelling approaches, and that the best LSTM models outperform it slightly. / I denna rapport analyserar vi sentimentet, eller attityden, i aktieanalysrapporter skrivna av analytiker på Skandinaviska Enskilda Banken (SEB). Etablerade statistiska metoder och maskininlärningsmetoder för klassificering av sentimentet i textdokument som antingen positivt eller negativt presenteras. Vi är speciellt intresserade av en typ av rekurrent neuronnät känt som long short-term memory (LSTM). Vidare undersöker vi två olika scheman för att märka upp träningsdatan som genereras från rapporterna. Riktmärken för klassificeringsgraden erhålls med hjälp av logistisk regression. Slutligen implementeras två olika ordrepresentationsmodeller och dubbelriktad LSTM av varierande nätverksstorlek, och jämförs med riktmärkena. Vi finner att logistisk regression presterar bra för ett av märkningsschemana, och att LSTM har något bättre prestanda.

[pt] Prever precisamente a produção de óleo é essencial para o planejamento e administração de um reservatório. Entretanto, prever a produção de óleo é um problema complexo e não linear, devido a todas as propriedades geofísicas que com pequenas variações podem resultar em differentes cenários. Além disso, todas as decisões tomadas durante a exploração do projeto devem considerar diferentes algoritmos para simular dados, fornecer cenários e conduzir a boas deduções. Para reduzir as incertezas nas simulações, estudos recentes propuseram o uso de algoritmos de aprendizado de maquina para solução de problemas da engenharia de reservatórios, devido a capacidade desses modelos de extrair o maxiomo de informações de um conjunto de dados. Essa tese propôe o uso ed duas tecnicas de machine learning para prever a produção diaria de óleo de um reservatório. Inicialmente, a produção diária de óleo é considerada uma série temporal, é pré-processada e reestruturada como um problema de aprendizado supervisionado. O modelo Random Forest, uma extensão das arvores de decisão muito utilizado em problemas de regressão e classificação, é utilizado para predizer um passo de tempo a frente. Entretanto, as restrições dessa abordagem nos conduziram a um modelo mais robusto, as redes neurais recorrentes LSTM, que são utilizadas em varios estudos como uma ferramenta dee aprendizado profundo adequada para modelagem de séries temporais. Várias configurações de redes LSTM foram construidas para implementar a previsão de um passo de tempo e de multiplos passos de tempo, a pressão do fundo de poço foi incorporada aos dados de entrada. Para testar a eficacia dos modelos propostos, foram usados quatro conjunto de dados diferentes, três gerados sintéticamente e um conjunto de dados reais do campo de produção VOlve, como casos de estudo para conduzir os experimentos. Os resultados indicam que o Random Forest é suficiente para previsões de um passo de tempo da produção de óleo e o LSTM é capaz de lidar com mais dados de entrada e estimar multiplos passos de tempo da produção de óleo. / [en] Precisely forecasting oil field performance is essential in oil reservoir planning and management. Nevertheless, forecasting oil production is a complex nonlinear problem due to all geophysical and petrophysical properties that may result in different effects with a bit of change. Thus, all decisions to be made during an exploitation project must consider different efficient algorithms to simulate data, providing robust scenarios to lead to the best deductions. To reduce the uncertainty in the simulation process, recent studies have efficiently introduced machine learning algorithms for solving reservoir engineering problems since they can extract the maximum information from the dataset. This thesis proposes using two machine learning techniques to predict the daily oil production of an offshore reservoir. Initially, the oil rate production is considered a time series and is pre-processed and restructured to fit a supervised learning problem. The Random Forest model is used to forecast a one-time step, which is an extension of decision tree learning, widely used in regression and classification problems for supervised machine learning. Regardless, the restrictions of this approach lead us to a more robust model, the LSTM RNN s, which are proposed by several studies as a suitable deep learning technique for time series modeling. Various configurations of LSTM RNN s were constructed to implement single-step and multi-step oil rate forecasting and down-hole pressure was incorporated to the inputs. For testing the robustness of the proposed models, we use four different datasets, three of them synthetically generated and one from a public real dataset, the Volve oil field, as a case study to conduct the experiments. The results indicate that the Random Forest model could sufficiently estimate the one-time step of the oil field production, and LSTM could handle more inputs and adequately estimate multiple-time steps of oil production.

Detta arbete handlar om att prediktera energiproduktion för en solcellsanläggning som är installerat på ett flerbostadshus i Karlstad. Syftet är att skapa en modell som tar data från föregående dygn som input och baserad på det  predikterar solenergiproduktionen 1h framåt. För att uppnå arbetets syfte presenteras teorier om maskininlärning och statistiska utvärderingsmetoder. Modellen implementeras med hjälp av programkod i toolboxen deep learning i MATLAB. Där tränas det på tidigare data för att hitta någon form av mönster och baserad på det prediktera energiproduktionen för 1h framåt givet energiproduktionen från föregående 24h. Algoritmen som används i arbetet är LSTM. I arbetet undersöks också hur mycket data bakåt i tiden som modellen behöver tränas på för att predikteringen ska ge en god indikation på energiproducering. Resultatet visar att bästa modell erhålls genom att träna modellen med 1års data bakåt i tiden med specifika nätverksinställningar . Vidare, ger modellen en MAE på 0.998kWh och en RMSE på 1.765kWh. / This project is about predicting energy production for a photovoltaic system that is installed on a building in Karlstad. The aim is to create a model that takes data from the previous day as input and based on that predicts solar energy production 1h ahead. To achieve the purpose of the work, theories on machine learning as well as statistical evaluation methods are presented. The model is implemented using program code in deep learning toolbox in MATLAB. There, it is trained on previous data to find some pattern in the data and based on that predict energy production for 1h ahead given the energy production from the previous 24h. The algorithm used in the project is LSTM. The project also examines how much data back in time the model needs to be trained on to give a good indication of solar energy production. The results show that the best model derives from training models with 1 year of historic data and with specific network settings. Furthermore, models provide an MAE of 0.998kWh and an RMSE of 1.765kWh.

The main thrust of time-series forecasting models in recent years has gone in the direction of pattern-based learning, in which the input variable for the models is a vector of past observations of the variable itself to predict. The most used models based on this traditional pattern-based approach are the autoregressive integrated moving average model (ARIMA) and long short-term memory neural networks (LSTM). The main drawback of the mentioned approaches is their inability to react when the underlying relationships in the data change resulting in a degrading predictive performance of the models. In order to solve this problem, various studies seek to incorporate external factors into the models treating the system as a black box using a machine learning approach which generates complex models that require a large amount of data for their training and have little interpretability. In this thesis, three different algorithms have been proposed to incorporate additional external factors into these pattern-based models, obtaining a good balance between forecast accuracy and model interpretability. After applying these algorithms in a study case of Ethereum price time-series forecasting, it is shown that the prediction error can be efficiently reduced by taking into account these influential external factors compared to traditional approaches while maintaining full interpretability of the model. / Huvudinstrumentet för prognosmodeller för tidsserier de senaste åren har gått i riktning mot mönsterbaserat lärande, där ingångsvariablerna för modellerna är en vektor av tidigare observationer för variabeln som ska förutsägas. De mest använda modellerna baserade på detta traditionella mönsterbaserade tillvägagångssätt är auto-regressiv integrerad rörlig genomsnittsmodell (ARIMA) och långa kortvariga neurala nätverk (LSTM). Den huvudsakliga nackdelen med de nämnda tillvägagångssätten är att de inte kan reagera när de underliggande förhållandena i data förändras vilket resulterar i en försämrad prediktiv prestanda för modellerna. För att lösa detta problem försöker olika studier integrera externa faktorer i modellerna som behandlar systemet som en svart låda med en maskininlärningsmetod som genererar komplexa modeller som kräver en stor mängd data för deras inlärning och har liten förklarande kapacitet. I denna uppsatsen har tre olika algoritmer föreslagits för att införliva ytterligare externa faktorer i dessa mönsterbaserade modeller, vilket ger en bra balans mellan prognosnoggrannhet och modelltolkbarhet. Efter att ha använt dessa algoritmer i ett studiefall av prognoser för Ethereums pristidsserier, visas det att förutsägelsefelet effektivt kan minskas genom att ta hänsyn till dessa inflytelserika externa faktorer jämfört med traditionella tillvägagångssätt med bibehållen full tolkbarhet av modellen.

The handwritten recognition system is a process of learning a pattern from a given image of text. The recognition process usually combines a computer vision task with sequence learning techniques. Transcribing texts from the scanned image remains a challenging problem, especially when the documents are highly degraded, or have excessive dusty noises. Nowadays, there are several handwritten recognition systems both commercially and in free versions, especially for Latin based languages. However, there is no prior study that has been built for Ge’ez handwritten ancient manuscript documents. In contrast, the language has many mysteries of the past, in human history of science, architecture, medicine and astronomy. In this thesis, we present two separate recognition systems. (1) A character-level recognition system which combines computer vision for character segmentation from ancient books and a vanilla Convolutional Neural Network (CNN) to recognize characters. (2) An end- to- end segmentation free handwritten recognition system using CNN, Multi-Dimensional Recurrent Neural Network (MDRNN) with Connectionist Temporal Classification (CTC) for the Ethiopic (Ge’ez) manuscript documents. The proposed character label recognition model outperforms 97.78% accuracy. In contrast, the second model provides an encouraging result which indicates to further study the language properties for better recognition of all the ancient books. / Det handskrivna igenkännings systemet är en process för att lära sig ett mönster från en viss bild av text. Erkännande Processen kombinerar vanligtvis en datorvisionsuppgift med sekvens inlärningstekniker. Transkribering av texter från den skannade bilden är fortfarande ett utmanande problem, särskilt när dokumenten är mycket försämrad eller har för omåttlig dammiga buller. Nuförtiden finns det flera handskrivna igenkänningar system både kommersiellt och i gratisversionen, särskilt för latin baserade språk. Det finns dock ingen tidigare studie som har byggts för Ge’ez handskrivna gamla manuskript dokument. I motsats till detta språk har många mysterier från det förflutna, i vetenskapens mänskliga historia, arkitektur, medicin och astronomi. I denna avhandling presenterar vi två separata igenkänningssystem. (1) Ett karaktärs nivå igenkänningssystem som kombinerar bildigenkänning för karaktär segmentering från forntida böcker och ett vanilj Convolutional Neural Network (CNN) för att erkänna karaktärer. (2) Ett änd-till-slut-segmentering fritt handskrivet igenkänningssystem som använder CNN, Multi-Dimensional Recurrent Neural Network (MDRNN) med Connectionist Temporal Classification (CTC) för etiopiska (Ge’ez) manuskript dokument. Den föreslagna karaktär igenkännings modellen överträffar 97,78% noggrannhet. Däremot ger den andra modellen ett uppmuntrande resultat som indikerar att ytterligare studera språk egenskaperna för bättre igenkänning av alla antika böcker.