As the real estate market plays a central role in a countries financial situation, as a life insurer, a bank and a property developer, Skandia wants a method for better assessing the risks connected to the real estate market. The goal of this paper is to increase the understanding of property market risk and its covariate risks and to conduct an analysis of how a fall in real estate prices could affect Skandia’s exposed assets.This paper explores a recurrent neural network model with the aim of quantifying identified risk factors using exogenous data. The recurrent neural network model is compared to a vector autoregressive model with exogenous inputs that represent economic conditions.The results of this paper are inconclusive as to which method that produces the most accurate model under the specified settings. The recurrent neural network approach produces what seem to be better results in out-of-sample validation but both the recurrent neural network model and the vector autoregressive model fail to capture the hypothesized relationship between the exogenous and modeled variables. However producing results that does not fit previous assumptions, further research into artificial neural networks and tests with additional variables and longer sample series for calibration is suggested as the model preconditions are promising.

The desire to follow student learning within intelligent tutoring systems in near real time has led to the development of several models anticipating the correctness of the next item as students work through an assignment. Such models have in- cluded Bayesian Knowledge Tracing (BKT), Performance Factors Analysis (PFA), and more recently with developments in Deep Learning, Deep Knowledge Tracing (DKT). The DKT model, based on the use of a recurrent neural network, exhibited promising results in paper [PBH+15]. Thus far, however, the model has only considered the knowledge components of the problems and correctness as input, neglecting the breadth of other features col- lected by computer-based learning platforms. This work seeks to improve upon the DKT model by incorporating more features at the problem-level and student-level. With this higher dimensional input, an adaption to the original DKT model struc- ture is also proposed, incorporating an Autoencoder network layer to convert the input into a low dimensional feature vector to reduce both the resource requirement and time needed to train. Experimental results show that our adapted DKT model, which includes more combinations of features, can effectively improve accuracy.

The recurrent neural networks have the ability to grasp the temporal patterns withinthe data. This is a property that can be used in order to help a recommender system bettertaking into account the user past history. Still the dimensionality problem that raiseswithin the recommender system field also raises here as the number of items the systemhave to be aware of is susceptibility high. Recent research have studied the use of such neural networks at a user’s session level.This thesis rather examines the use of this technique at a whole user’s past history levelassociated with techniques such as embeddings and softmax sampling in order to accommodatewith the high dimensionality. The proposed method results in a sequence prediction model that can be used as is forthe recommender task or as a feature within a more complex system. / De Recurrent Neural Networks har möjlighet att förstå de tidsmässiga mönstren inom data. Det här är en egenskap som kan användas för att hjälpa ett rekommendatörsystem bättre med hänsyn till användarens historia. Problemet med dimensioner inom rekommendatörsystem uppstår dock även här, eftersom antalet saker som systemet måste vara medveten om är extremt många. Nyare forskning har studerat användningen av sådana neurala nätverk på en användaressessionsnivå. Denna avhandling undersöker snarare användningen av denna teknik som en hel användares tidigare historiknivå i samband med tekniker som inbäddning och softmax-provtagning för att tillgodose den höga dimensionen. Den föreslagna metoden resulterar i en sekvensprediktionsmodell som kan användas som för recommender-uppgiften eller som en funktion inom ett mer komplext system.

The purpose of this project is to evaluate the performance of a forecasting model based on a multivariate dataset consisting of time series of traffic characteristic performance data from a mobile network. The forecasting is made using machine learning with a deep neural network. The first part of the project involves the adaption of the model design to fit the dataset and is followed by a number of simulations where the aim is to tune the parameters of the model to give the best performance. The simulations show that with well tuned parameters, the neural network performes better than the baseline model, even when using only a univariate dataset. If a multivariate dataset is used, the neural network outperforms the baseline model even when the dataset is small.

Les explosions combinées de la puissance computationnelle et de la quantité de données disponibles ont fait des algorithmes les nouveaux facteurs limitants en machine learning. L’objectif de cette thèse est donc d’introduire de nouvelles méthodes capables de tirer profit de quantités de données et de ressources computationnelles importantes. Nous présentons deux contributions indépendantes. Premièrement, nous développons des algorithmes d’optimisation rapides, adaptés aux avancées en architecture de calcul parallèle pour traiter des quantités massives de données. Nous introduisons un cadre d’analyse pour les algorithmes parallèles asynchrones, qui nous permet de faire des preuves correctes et simples. Nous démontrons son utilité en analysant les propriétés de convergence et d’accélération de deux nouveaux algorithmes. Asaga est une variante parallèle asynchrone et parcimonieuse de Saga, un algorithme à variance réduite qui a un taux de convergence linéaire rapide dans le cas d’un objectif lisse et fortement convexe. Dans les conditions adéquates, Asaga est linéairement plus rapide que Saga, même en l’absence de parcimonie. ProxAsaga est une extension d’Asaga au cas plus général où le terme de régularisation n’est pas lisse. ProxAsaga obtient aussi une accélération linéaire. Nous avons réalisé des expériences approfondies pour comparer nos algorithms à l’état de l’art. Deuxièmement, nous présentons de nouvelles méthodes adaptées à la prédiction structurée. Nous nous concentrons sur les réseaux de neurones récurrents (RNNs), dont l’algorithme d’entraînement traditionnel – basé sur le principe du maximum de vraisemblance (MLE) – présente plusieurs limitations. La fonction de coût associée ignore l’information contenue dans les métriques structurées ; de plus, elle entraîne des divergences entre l’entraînement et la prédiction. Nous proposons donc SeaRNN, un nouvel algorithme d’entraînement des RNNs inspiré de l’approche dite “learning to search”. SeaRNN repose sur une exploration de l’espace d’états pour définir des fonctions de coût globales-locales, plus proches de la métrique d’évaluation que l’objectif MLE. Les modèles entraînés avec SeaRNN ont de meilleures performances que ceux appris via MLE pour trois tâches difficiles, dont la traduction automatique. Enfin, nous étudions le comportement de ces modèles et effectuons une comparaison détaillée de notre nouvelle approche aux travaux de recherche connexes. / The impressive breakthroughs of the last two decades in the field of machine learning can be in large part attributed to the explosion of computing power and available data. These two limiting factors have been replaced by a new bottleneck: algorithms. The focus of this thesis is thus on introducing novel methods that can take advantage of high data quantity and computing power. We present two independent contributions. First, we develop and analyze novel fast optimization algorithms which take advantage of the advances in parallel computing architecture and can handle vast amounts of data. We introduce a new framework of analysis for asynchronous parallel incremental algorithms, which enable correct and simple proofs. We then demonstrate its usefulness by performing the convergence analysis for several methods, including two novel algorithms. Asaga is a sparse asynchronous parallel variant of the variance-reduced algorithm Saga which enjoys fast linear convergence rates on smooth and strongly convex objectives. We prove that it can be linearly faster than its sequential counterpart, even without sparsity assumptions. ProxAsaga is an extension of Asaga to the more general setting where the regularizer can be non-smooth. We prove that it can also achieve a linear speedup. We provide extensive experiments comparing our new algorithms to the current state-of-art. Second, we introduce new methods for complex structured prediction tasks. We focus on recurrent neural networks (RNNs), whose traditional training algorithm for RNNs – based on maximum likelihood estimation (MLE) – suffers from several issues. The associated surrogate training loss notably ignores the information contained in structured losses and introduces discrepancies between train and test times that may hurt performance. To alleviate these problems, we propose SeaRNN, a novel training algorithm for RNNs inspired by the “learning to search” approach to structured prediction. SeaRNN leverages test-alike search space exploration to introduce global-local losses that are closer to the test error than the MLE objective. We demonstrate improved performance over MLE on three challenging tasks, and provide several subsampling strategies to enable SeaRNN to scale to large-scale tasks, such as machine translation. Finally, after contrasting the behavior of SeaRNN models to MLE models, we conduct an in-depth comparison of our new approach to the related work.

The demand for high data rates communication and scarcity of spectrum in existing microwave bands has been the key aspect in 5G. To fulfill these demands, the millimeter wave (mmWave) with large bandwidths has been proposed to enhance the efficiency and the stability of the 5G network. In mmWave communication, the concentration of the transmission signal from the antenna is conducted by beamforming and beam tracking. However, state-of-art methods in beam tracking lead to high resource consumption. To address this problem, we develop 2 machine-learning-based solutions for overhead reduction. In this paper, a scenario configuration simulator is proposed as the data collection approach. Several LSTM based time series prediction models are trained for experiments. Since the overhead is reduced by decreasing the number of sweeping beams in solutions, multiple data imputation methods are proposed to improve the performance of the solution. These methods are based on Multiple Imputation by Chained Equations (MICE) and generative adversarial networks. Both qualitative and quantitative experimental results on several types of datasets demonstrate the efficacy of our solution. / Efterfrågan på hög datahastighetskommunikation och brist på spektrum i befintliga mikrovågsband har varit nyckelaspekten i 5G. För att uppfylla dessa krav har millimetervåg (mmWave) med stora bandbredder föreslagits för att förbättra effektiviteten och stabiliteten i 5G-nätverket. I mmWavekommunikation utförs koncentrationen av överföringssignalen från antennen genom strålformning och strålspårning. Toppmoderna metoder inom strålspårning leder dock till hög resursförbrukning. För att lösa detta problem utvecklar vi två maskininlärningsbaserade lösningar för reduktion av omkostnader. I det här dokumentet föreslås en scenariokonfigurationssimulator som datainsamlingsmetod. Flera LSTM-baserade modeller för förutsägelse av tidsserier tränas för experiment. Eftersom omkostnaderna reduceras genom att minska svepstrålarna i lösningar föreslås flera datainputeringsmetoder för att förbättra lösningens prestanda. Dessa metoder är baserade på Multipel Imputation by Chained Equations (MICE) och generativa kontroversiella nätverk. Både kvalitativa och kvantitativa experimentella resultat på flera typer av datamängder visar effektiviteten i vår lösning.

Action detection is an attractive area for researchers in computer vision, healthcare, physiotherapy, psychology, and others. Intensive work has been done in this area due to its wide range of applications such as security surveillance, video tagging, Human-Computer Interaction (HCI), robotics, medical diagnosis, sports analysis, interactive gaming, and many others. After the deep learning booming results in computer vision tasks like image classification, many researchers have tried to extend the success of deep learning models to video classification and activity recognition. The research question of this thesis is to study the use of the 2D human poses extracted by a DNN-based model from RGB frames only, for the online activity detection task and comparing it with the state of the art solutions that utilize the human 3D skeletal data extracted by a depth sensor as an input. At the same time, this work showed the importance of input pre-processing and filtering on improving the performance of the online human activity detector. Detecting gym exercises and counting the repetitions in real-time using the human skeletal data versus the 2D poses have been studied in-depth in this work. The contributions of this work are as follows: 1) generating RGB-D dataset for a set of gym exercises, 2) proposing a novel real-time skeleton-based Double Representational RNN (DR-RNN) network architecture for the online action detection, 3) Demonstrating the ability of the proposed model to achieve satisfiable results using pose estimation models applied on RGB frames, 4) introducing a novel learnable exponential filter for the online low latency filtering applications.

Long short-term memory (LSTM) neural networks have been proven to be effective for time series prediction, even in some instances where the data is non-stationary. This lead us to examine their predictive ability of stock market returns, as the development of stock prices and returns tend to be a non-stationary time series. We used daily stock trading data to let an LSTM train models at predicting daily returns for 60 stocks from the OMX30 and Nasdaq-100 indices. Subsequently, we measured their accuracy, precision, and recall. The mean accuracy was 49.75 percent, meaning that the observed accuracy was close to the accuracy one would observe by randomly selecting a prediction for each day and lower than the accuracy achieved by blindly predicting all days to be positive. Finally, we concluded that further improvements need to be made for models trained by LSTMs to have any notable predictive ability in the area of stock returns.

This diploma thesis describes methods suitable for automatic detection of events from video sequences focused on football matches. The first part of the work is focused on the analysis and creation of procedures for extracting informations from available data. The second part deals with the implementation of selected methods and neural network algorithm for corner kick detection. Two experiments were performed in this work. The first captures static information from one image and the second is focused on detection from spatio-temporal data. The output of this work is a program for automatic event detection, which can be used to interpret the results of the experiments. This work may figure as a basis to gain new knowledge about the issue and also to the further development of detection events from football.

Intelligent transportation systems (ITS) are essential tools for traffic planning, analysis, and forecasting that can utilize the huge amount of traffic data available nowadays. In this work, we aggregated detailed traffic flow sensor data, Waze reports, OpenStreetMap (OSM) features, and weather data, from California Bay Area for 6 months. Using that data, we studied three novel ITS applications using convolutional neural networks (CNNs) and recurrent neural networks (RNNs). The first experiment is an analysis of the relation between roadway shapes and accident occurrence, where results show that the speed limit and number of lanes are significant predictors for major accidents on highways. The second experiment presents a novel method for forecasting congestion severity using crowdsourced data only (Waze, OSM, and weather), without the need for traffic sensor data. The third experiment studies the improvement of traffic flow forecasting using accidents, number of lanes, weather, and time-related features, where results show significant performance improvements when the additional features where used.