京都大学 / 0048 / 新制・課程博士 / 博士(農学) / 甲第22077号 / 農博第2369号 / 新制||農||1072(附属図書館) / 学位論文||R1||N5231(農学部図書室) / 京都大学大学院農学研究科地域環境科学専攻 / (主査)教授 近藤 直, 教授 清水 浩, 教授 飯田 訓久 / 学位規則第4条第1項該当 / Doctor of Agricultural Science / Kyoto University / DGAM

This diploma thesis deals with the detection of fire and smoke from the image signal. The approach of this work uses a combination of convolutional and recurrent neural network. Machine learning models created in this work contain inception modules and blocks of long short-term memory. The research part describes selected models of machine learning used in solving the problem of fire detection in static and dynamic image data. As part of the solution, a data set containing videos and still images used to train the designed neural networks was created. The results of this approach are evaluated in conclusion.

High-performance computing (HPC) systems continue growing in both scale and complexity. These large-scale, heterogeneous systems generate tens of millions of log messages every day. Effective log analysis for understanding system behaviors and identifying system anomalies and failures is highly challenging. Existing log analysis approaches use line-by-line message processing. They are not effective for discovering subtle behavior patterns and their transitions, and thus may overlook some critical anomalies. In this dissertation research, I propose a system log event block detection (SLEBD) method which can extract the log messages that belong to a component or system event into an event block (EB) accurately and automatically. At the event level, we can discover new event patterns, the evolution of system behavior, and the interaction among different system components. To find critical event sequences, existing sequence mining methods are mostly based on the a priori algorithm which is compute-intensive and runs for a long time. I develop a novel, topology-aware sequence mining (TSM) algorithm which is efficient to generate sequence patterns from the extracted event block lists. I also train a long short-term memory (LSTM) model to cluster sequences before specific events. With the generated sequence pattern and trained LSTM model, we can predict whether an event is going to occur normally or not. To accelerate such predictions, I propose a design flow by which we can convert recurrent neural network (RNN) designs into register-transfer level (RTL) implementations which are deployed on FPGAs. Due to its high parallelism and low power, FPGA achieves a greater speedup and better energy efficiency compared to CPU and GPU according to our experimental results.

This thesis investigates how to remove personal data from logs using machine learning when rule-based scripts are not enough and manual scanning is too extensive. Three types of machine learning models were created and compared. One word model using logistic regression, another word model using LSTM and a sentence model also using LSTM. Data logs were cleaned and annotated using rule-based scripts, datasets from various countries and dictionaries from various languages. The created dataset for the sentence based model was imbalanced, and a lite version of data augmentation was applied. A hyperparameter optimization library was used to find the best hyperparameter combination. The models learned the training and the validation set well but did perform worse on the test set consisting of log data from a different server logging other types of data. / Detta examensarbete undersöker om det är möjligt att skapa ett program som automatiskt identifierar och tar bort persondata från dataloggar med hjälp av maskinlärning. Att förstå innebörden av vissa ord kräver också kontext: Banan kan syfta på en banan som man kan äta eller en bana som man kan springa på. Kan en maskinlärningsmodell ta nytta av föregående och efterkommande ord i en sekvens av ord för att få en bättre noggrannhet på om ordet är känsligt eller ej. Typen av data som förekommer i loggarna kan vara bland annat namn, personnummer, användarnamn och epostadress. För att modellen ska kunna lära sig att känna igen datan krävs det att det finns data som är färdigannoterad med facit i hand. Telefonnummer, personnummer och epostadress kan bara se ut på ett visst sätt och behöver nödvändigtvis ingen maskininlärning för att kunna pekas ut. Kan man skapa en generell modell som fungerar på flera typer av dataloggar utan att använda regelbaserade algoritmer. Resultaten visar att den annoterade datan som användes för träning kan ha skiljt allt för mycket från de loggar som har testats på (osedd data), vilket betyder att modellen inte är bra på att generalisera.

Nowadays, the ideas of continuous integration and continuous delivery are under heavy usage in order to achieve rapid software development speed and quick product delivery to the customers with good quality. During the process ofmodern software development, the testing stage has always been with great significance so that the delivered software is meeting all the requirements and with high quality, maintainability, sustainability, scalability, etc. The key assignment of software testing is to find bugs from every test and solve them. The developers and test engineers at Ericsson, who are working on a large scale software architecture, are mainly relying on the logs generated during the testing, which contains important information regarding the system behavior and software status, to debug the software. However, the volume of the data is too big and the variety is too complex and unpredictable, therefore, it is very time consuming and with great efforts for them to manually locate and resolve the bugs from such vast amount of log data. The objective of this thesis project is to explore a way to conduct log analysis efficiently and effectively by applying relevant machine learning algorithms in order to help people quickly detect the test failure and its possible causalities. In this project, a method of preprocessing and clusering original logs is designed and implemented in order to obtain useful data which can be fed to machine learning algorithms. The comparable log analysis, based on two machine learning algorithms - Recurrent Neural Network and Naive Bayes, is conducted for detecting the place of system failures and anomalies. Finally, relevant experimental results are provided and analyzed.

Recommendation systems become more and more necessary due to the growing supply chain. Therefore, scientists are developing models that can serve different recommendation needs faster than before, and it is getting more complicated to choose the model for a specific case. In this thesis, there are three neural collaborative filtering methods compared regarding dataset fit. This research shows that there is no one-fits-all method. There is much space for improvement in all the areas: dataset selection and aggregation, method development and operation, and selective approaches for the analysis of the results. In the thesis, three contrasting datasets are chosen (Chess, Library, and LastFM), and three novel approaches are tested: recently released Dynamic Graph Collaborative Filtering (DGCF) and Dynamic Embeddings for Interaction Prediction (DeePRed) are compared to the Joint Dynamic User- Item Embeddings (JODIE) as the baseline. Results show DeePRed being a state-of-the-art model that outperforms other methods. It runs an epoch for a small dataset in less than a minute, shows great prediction accuracy in an average of 98% for small datasets. However, DGCF does not show accuracy improvement over JODIE but is significantly faster for an extensive dataset. / Rekommendationssystem blir mer och mer nödvändiga på grund av den växande försörjningskedjan. Därför utvecklar forskare modeller som kan tjäna olika rekommendationsbehov snabbare än tidigare och det blir mer och mer komplicerat att välja modell för ett specifikt fall. I denna avhandling finns det tre neurologiska samarbetsfiltreringsmetoder som jämförs avseende deras gran för olika datamängder. Denna forskning visar att det inte finns någon metod som passar alla och det finns mycket utrymme för förbättring inom alla områden: datasatsval och aggregering, metodutveckling och drift och selektiva metoder för analys av resultaten. I avhandlingen väljs tre kontrasterande datamängder (Chess, Library och LastFM) och tre nya metoder testas: nyligen släppt Dynamic Graph Collaborativefiltering (DGCF) och Dynamic Embedding for Interaction Prediction (DeePRed) jämförs med Joint Dynamic User-Item. Inbäddning (JODIE) som baslinje. Resultaten visar att (DeePRed) är en avancerad modell som överträffar andra metoder som snabba genom att köra en epok för liten dataset på mindre än en minut, vilket visar stor förutsägelsesnoggrannhet i genomsnitt 98% för små datamängder. Men (DGCF) visar inte förbättring av noggrannhet jämfört med (JODIE), men är betydligt snabbare för en stor dataset.

Natural language processing has many important applications in today, such as translations, spam filters, and other useful products. To achieve these applications supervised and unsupervised machine learning models, have shown to be successful. The most important aspect of these models is what the model can achieve with different datasets. This article will examine how RNN models compare with Naive Bayes in text classification. The chosen RNN models are long short-term memory (LSTM) and gated recurrent unit (GRU). Both LSTM and GRU will be trained using the flair Framework. The models will be trained on three separate datasets with different compositions, where the trend within each model will be examined and compared with the other models. The result showed that Naive Bayes performed better on classifying short sentences than the RNN models, but worse in longer sentences. When trained on a small dataset LSTM and GRU had a better result then Naive Bayes. The best performing model was Naive Bayes, which had the highest accuracy score in two out of the three datasets.

Bipolar mood disorder is a severe mental condition that has multiple episodesof either of two types: manic or depressive. These phases can lead patients tobecome hyperactive, hyper-sexual, lethargic, or even commit suicide — all ofwhich seriously impair the quality of life for patients. Predicting these phaseswould help patients manage their lives better and improve our ability to applymedical interventions. Traditionally, interviews are conducted in the evening topredict potential episodes in the following days. While machine learningapproaches have been used successfully before, the data was limited tomeasuring a few self-reported parameters each day. Using biometrics recordedat short intervals over many months presents a new opportunity for machinelearning approaches. However, phases of unrest and hyperactivity, which mightbe predictive signals, are not only often experienced long before the onset ofmanic or depressive phases but are also separated by several uneventful days.This delay and its aperiodic occurrence are a challenge for deep learning. In thisthesis, a fictional dataset that mimics long and irregular delays is created andused to test the effects of such long delays and rare events. LSTMs, RNNs, andGRUs are the go-to models for deep learning in this situation. However, theydiffer in their ability to be trained over a long time. As their acronym suggests,LSTMS are believed to be easier to train and to have a better ability to remember(as their name suggests) than their simpler RNN counterparts. GRUs representa compromise in complexity between RNNs and LSTMs. Here, I will show that,contrary to the common assumption, LSTMs are surprisingly forgetful and thatRNNs have a much better ability to generalize over longer delays with shortersequences. At the same time, I could confirm that LSTMs are easily trained ontasks that have more prolonged delays.

The advent of recent high throughput sequencing technologies resulted in an unexplored big data of genomics and transcriptomics that might help to answer various research questions in Parkinson’s disease(PD) progression. While the literature has revealed various predictive models that use longitudinal clinical data for disease progression, there is no predictive model based on RNA-Sequence data of PD patients. This study investigates how to predict the PD Progression for a patient’s next medical visit by capturing longitudinal temporal patterns in the RNA-Seq data. Data provided by Parkinson Progression Marker Initiative (PPMI) includes 423 PD patients with a variable number of visits for a period of 4 years. We propose a predictive model based on a Recurrent Neural Network (RNN) with dense connections. The results show that the proposed architecture is able to predict PD progression from high dimensional RNA-seq data with a Root Mean Square Error (RMSE) of 6.0 and rank-order correlation of (r=0.83, p<0.0001) between the predicted and actual disease status of PD. We show empirical evidence that the addition of dense connections and batch normalization into RNN layers boosts its training and generalization capability.

The demand for accurate localization under indoor environments has increased dramatically in recent years. To be cost-effective, most of the localization solutions are based on the WiFi signals, utilizing the pervasive deployment of WiFi infrastructure and availability of the WiFi enabled mobile devices. In this thesis, we develop completed indoor localization solutions based on WiFi fingerprinting and machine learning approaches with two types of WiFi fingerprints including received signal strength indicator (RSSI) and channel state information (CSI). Starting from the low complexity algorithm, we propose a soft range limited K nearest neighbours (SRL-KNN) to address spatial ambiguity and the fluctuation of WiFi signals. SRL-KNN exploits RSSI and scales the fingerprint distance by a range factor related to the physical distance between the user’s previous position and the reference location in the database. Although utilizing the prior locations, SRL-KNN does not require knowledge of the exact moving speed and direction of the user. Besides, the idea of the soft range limiting factor can be applied to all of the existed probabilistic methods, i.e., parametric and nonparametric methods, to improve their performances. A semi-sequential short term memory step is proposed to add to the existed probabilistic methods to reduce their spatial ambiguity of fingerprints and boost significantly their localization accuracy. In the following research phase, instead of locating user's position one at a time as in the cases of conventional algorithms, our recurrent neuron networks (RNNs) solution aims at trajectory positioning and takes into account of the relation among RSSI measurements in a trajectory. The results using different types of RNN including vanilla RNN, long short-term memory (LSTM), gated recurrent unit (GRU) and bidirectional LSTM (BiLSTM) are presented. Next, the problem of localization using only one single router is analysed. CSI information will be adopted along with RSSI to enhance the localization accuracy. Each of the reference point (RP) is presented by a group of CSI measurements from several WiFi subcarriers which we call CSI images. The combination of convolutional neural network (CNN) and LSTM model is proposed. CNN extracts the useful information from several CSI values (CSI images), and then LSTM will exploit this information in sequential timesteps to determine the user's location. Finally, a fully practical passive indoor localization is proposed. Most of the conventional methods rely on the collected WiFi signal on the mobile devices (active information), which requires a dedicated software to be installed. Different from them, we leverage the received data of the routers (passive information) to locate the position of the user. The localization accuracy is investigated through experiments with several phones, e.g., Nexus 5, Samsung, Iphone and HTC, in hundreds of testing locations. The experimental results demonstrate that our proposed localization scheme achieves an average localization error of around 1.5 m when the phone is in idle mode, and approximately 1 m when it actively transmits data. / Graduate