Due to the growing share of renewable energy in countries' power systems, the need for precise forecasting of electricity consumption will increase. This paper considers two different approaches to time series forecasting, autoregressive moving average (ARMA) models and recurrent neural networks (RNNs). These are applied to Swedish electricity consumption data, with the aim of deriving simple yet efficient predictors. An additional aim is to analyse the impact of day of week and temperature on forecast accuracy. The models are evaluated on both long- and mid-term forecasting horizons, ranging from one day to one month. The results show that neural networks are superior for this task, although stochastic seasonal ARMA models also perform quite well. Including external variables only marginally improved the ARMA predictions, and had somewhat unclear effects on the RNN forecasting accuracy. Depending on the network model used, adding external variables had either a slightly positive or slightly negative impact on prediction accuracy.

The growing interest in more electrified vehicles has been pushing the industry and academia to pursue new and more accurate ways to estimate the xEV batteries State-of-Charge (SOC). The battery system still represents one of the many technical barriers that need to be eliminated or reduced to enable the proliferation of more xEV in the market, which in turn can help reduce CO2 emissions. Battery modelling and SOC estimation of Lithium-ion batteries (Li-ion) at a wide temperature range, including negative temperatures, has been a challenge for many engineers. For SOC estimation, several models configurations and approaches were developed and tested as results of this work, including different non-recurrent neural networks, such as Feedforward deep neural networks (FNN) and recurrent neural networks based on long short-term memory recurrent neural networks (LSTM-RNN). The approaches have considerably improved the accuracy presented in the previous state-of-the-art. They have expanded the application throughout five different Li-ion at a wide temperature range, achieving error as low as 0.66% Root Mean Square Error at -10⁰C using an FNN approach and 0.90% using LSTM-RNN. Therefore, the use of deep neural networks developed in this work can increase the potential for xEV application, especially where accuracy at negative temperatures is essential. For Li-ion modelling, a cell model using LSTM-RNN (LSTM-VM) was developed for the first time to estimate the battery cell terminal voltage and is compared against a gated recurrent unit (GRU-VM) approach and a Third-order Equivalent Circuit Model based on Thevenin theorem (ECM). The models were extensively compared for different Li-ion at a wide range of temperature conditions. The LSTM-VM has shown to be more accurate than the two other benchmarks, where could achieve 43 (mV) Root Mean Square Error at -20⁰C, a third when compared to the same situation using ECM. Although the difference between LSTM-VM and GRU-VM is not that steep. Finally, throughout the work, several methods to improve robustness, accuracy and training time have been introduced, including Transfer Learning applied to the development of SOC estimation models, showing great potential to reduce the amount of data necessary to train LSTM-RNN as well as improve its accuracy. / Thesis / Doctor of Philosophy (PhD) / For electric vehicle State-of-Charge estimation, several models configurations and approaches were developed and tested as results of this work, including different non-recurrent neural networks, such as Feedforward deep neural networks (FNN) and recurrent neural networks based on long short-term memory recurrent neural networks (LSTM-RNN). The approaches have considerably improved the accuracy presented in the previous state-of-the-art. They have expanded the application throughout five different Li-ion at a wide temperature range, achieving error as low as 0.66% Root Mean Square Error at -10⁰C using an FNN approach and 0.90% using LSTM-RNN. Therefore, the use of deep neural networks developed in this work can increase the potential for xEV application, especially where accuracy at negative temperatures is essential. For Li-ion modelling, a cell model using LSTM-RNN (LSTM-VM) was developed for the first time to estimate the battery cell terminal voltage and is compared against a gated recurrent unit (GRU-VM) approach and a Third-order Equivalent Circuit Model based on Thevenin theorem (ECM). The models were extensively compared for different Li-ion at a wide range of temperature conditions. The LSTM-VM has shown to be more accurate than the two other benchmarks, where could achieve 43 (mV) Root Mean Square Error at -20⁰C, a third when compared to the same situation using ECM. Although the difference between LSTM-VM and GRU-VM is not that steep.

No description available.

Les primates doivent pouvoir reconnaître de nouvelles situations pour pouvoir s'y adapter. La représentation de ces situations dans l'activité du cortex est le sujet de cette thèse. Les situations complexes s'expliquent souvent par l'interaction entre des informations sensorielles, internes et motrices. Des activités unitaires dénommées sélectivité mixte, qui sont très présentes dans le cortex préfrontal (CPF), sont un mécanisme possible pour représenter n'importe quelle interaction entre des informations. En parallèle, le Reservoir Computing a démontré que des réseaux récurrents ont la propriété de recombiner des entrées actuelles et passées dans un espace de plus haute dimension, fournissant ainsi un pré-codage potentiellement universel de combinaisons pouvant être ensuite sélectionnées et utilisées en fonction de leur pertinence pour la tâche courante. En combinant ces deux approches, nous soutenons que la nature fortement récurrente de la connectivité locale du CPF est à l'origine d'une forme dynamique de sélectivité mixte. De plus, nous tentons de démontrer qu'une simple régression linéaire, implémentable par un neurone seul, peut extraire n'importe qu'elle information/contingence encodée dans ces combinaisons complexes et dynamiques. Finalement, les entrées précédentes, qu'elles soient sensorielles ou motrices, à ces réseaux du CPF doivent être maintenues pour pouvoir influencer les traitements courants. Nous soutenons que les représentations de ces contextes définis par ces entrées précédentes doivent être exprimées explicitement et retournées aux réseaux locaux du CPF pour influencer les combinaisons courantes à l'origine de la représentation des contingences / In order to adapt to new situations, primates must be able to recognize these situations. How the cortex represents contingencies in its activity is the main subject of this thesis. First, complex new situations are often explained by the interaction between sensory, internal and motor information. Recent studies have shown that single-neuron activities referred to as mixed selectivity which are ubiquitous in the prefrontal cortex (PFC) are a possible mechanism to represent arbitrary interaction between information defining a contingency. In parallel, a recent area of reasearch referred to as Reservoir Computing has demonstrated that recurrent neural networks have the property of recombining present and past inputs into a higher dimensional space thereby providing a pre-coding of an essentially universal set of combinations which can then be selected and used arbitrarily for their relevance to the task at hand. Combining these two approaches we argue that the highly recurrent nature of local prefrontal connectivity is at the origin of dynamic form of mixed selectivity. Also, we attempt to demonstrate that a simple linear regression, implementable by a single neuron, can extract any information/ contingency encoded in these highly complex and dynamic combinations. In addition, previous inputs, whether sensory or motor, to these PFC networks must be maintained in order to influence current processing and behavioral demand. We argue that representations of contexts defined by these past inputs must be expressed explicitely and fed back to the local PFC networks in order to influence the current combinations at the origin of contingencies representation

No description available.

The goal of Intelligent Tutoring systems (ITSs) is to engage the students in sustained reasoning activity and to interact with students based on a deep understanding of student behavior. In order to understand student behavior, ITSs rely on student modeling methods to observes student actions in the tutor and creates a quantitative representation of student knowledge, interests, affective states. Good student models are going to effectively help ITSs customize instructions, engage student's interest and then promote learning. Thus, the work of building ITSs and advancing student modeling should be considered as two interconnected components of one system rather than two separate topics. In this work, we utilized the theoretical support of a well-known learning science theory, the spacing effect, to guide the development of an ITS, called Automatic Reassessment and Relearning System (ARRS). ARRS not only validated the effectiveness of spacing effect, but it also served as a testing field which allowed us to find out new approaches to improve student learning by conducting large-scale randomized controlled trials (RCTs). The rich data set we gathered from ARRS has advanced our understanding of robust learning and helped us build student models with advanced data mining methods. At the end, we designed a set of API that supports the development of ARRS in next generation ASSISTments platform and adopted deep learning algorithms to further improve retention performance prediction. We believe our work is a successful example of combining theory and practice to advance science and address real- world problems.

Recurrent neural networks (RNNs) are state of the art sequential machine learning tools, but have difficulty learning sequences with long-range dependencies due to the exponential growth or decay of gradients backpropagated through the RNN. Some methods overcome this problem by modifying the standard RNN architecure to force the recurrent weight matrix W to remain orthogonal throughout training. The first half of this thesis presents a novel orthogonal RNN architecture that enforces orthogonality of W by parametrizing with a skew-symmetric matrix via the Cayley transform. We present rules for backpropagation through the Cayley transform, show how to deal with the Cayley transform's singularity, and compare its performance on benchmark tasks to other orthogonal RNN architectures. The second half explores two deep learning approaches to problems in RNA secondary structure inference and compares them to a standard structure inference tool, the nearest neighbor thermodynamic model (NNTM). The first uses RNNs to detect paired or unpaired nucleotides in the RNA structure, which are then converted into synthetic auxiliary data that direct NNTM structure predictions. The second method uses recurrent and convolutional networks to directly infer RNA base pairs. In many cases, these approaches improve over NNTM structure predictions by 20-30 percentage points.

Orientador: Aparecido Augusto de Carvalho / Abstract: In the last few years, several studies have been carried out showing that neuromuscular electrical stimulation (NMES) can produce good therapeutic results in patients with spinal cord injury (SCI). This research introduces a new robust and intelligent control-based methodology for human lower limb rehabilitation via NMES using a continuous-time control technique named robust integral of the sign of the error (RISE). Although in the literature the RISE controller has shown good results without any fine-tuning method, a trial and error approach would quickly lead to muscle fatigue in SCI patients. Therefore, it was shown in this study that the control performance for robustly tracking a reference signal can be improved through the proposed approach by providing an intelligent tuning for each voluntary. Simulation results with a mathematical model and eight identified subjects from the literature are provided, and real experiments are performed with seven healthy and two paraplegic subjects. Besides, this research introduces the application of deep and dynamic neural networks namely the multilayer perceptron, a simple recurrent neural network, and the Long Short-Term memory architecture, to identify the nonlinear and time-varying relationship between the supplied NMES and achieved angular position. Identification results indicate good fitting to data and very low mean square error using few data for training, proving to be very prospective methods for proposing control-oriented ... (Complete abstract click electronic access below) / Resumo: Nos últimos anos, vários estudos foram realizados mostrando que a estimulação elétrica neuromuscular (EENM) pode produzir bons resultados terapêuticos em pacientes com lesão medular (LM). Esta pesquisa introduz uma nova metodologia robusta e inteligente baseada em controle para a reabilitação de membros inferiores humanos via EENM usando uma técnica de controle de tempo contínuo chamada robust integral of the sign of the error (RISE). Embora na literatura o controlador RISE tem demonstrado bons resultados sem qualquer método de ajuste fino, uma abordagem de tentativa e erro poderia levar rapidamente à fadiga muscular em pacientes com LM. Portanto, foi mostrado nesse estudo que o desempenho do controle para rastrear com robustez um sinal de referência pode ser melhorado através da abordagem proposta, fornecendo um ajuste inteligente para cada voluntário. Resultados de simulação com um modelo matemático e oito sujeitos identificados da literatura são fornecidos, e experimentos reais são feitos com sete indivíduos saudáveis ​​e dois paraplégicos. Além disso, esta pesquisa introduz a aplicação de redes neurais profundas e dinâmicas, especificamente o perceptron multicamadas, uma rede neural recorrente simples e a arquitetura Long Short-Term Memory, para identificar a relação não-linear e variante no tempo entre a EENM fornecida e a posição angular alcançada. Os resultados de identificação indicam boa adaptação aos dados e erro quadrático médio muito baixo usando poucos dados para... (Resumo completo, clicar acesso eletrônico abaixo) / Mestre

Statistical shift-reduce parsing involves the interplay of representation learning, structured learning, and inexact search. This dissertation considers approaches that tightly integrate these three elements and explores three novel models for shift-reduce CCG parsing. First, I develop a dependency model, in which the selection of shift-reduce action sequences producing a dependency structure is treated as a hidden variable; the key components of the model are a dependency oracle and a learning algorithm that integrates the dependency oracle, the structured perceptron, and beam search. Second, I present expected F-measure training and show how to derive a globally normalized RNN model, in which beam search is naturally incorporated and used in conjunction with the objective to learn shift-reduce action sequences optimized for the final evaluation metric. Finally, I describe an LSTM model that is able to construct parser state representations incrementally by following the shift-reduce syntactic derivation process; I show expected F-measure training, which is agnostic to the underlying neural network, can be applied in this setting to obtain globally normalized greedy and beam-search LSTM shift-reduce parsers.

L’ère numérique change de plus en plus le monde en raison de la quantité de données récoltées chaque jour. Le domaine médical est fortement affecté par cette explosion, car l’exploitation de ces données est un véritable atout pour l’aide à la pratique médicale. Dans cette thèse, nous proposons d’utiliser les vidéos chirurgicales dans le but de créer un système de chirurgie assistée par ordinateur. Nous nous intéressons principalement à reconnaître les gestes chirurgicaux à chaque instant afin de fournir aux chirurgiens des recommandations et des informations pertinentes. Pour ce faire, l’objectif principal de cette thèse est de reconnaître les outils chirurgicaux dans les vidéos de chirurgie de la cataracte. Dans le flux vidéo du microscope, ces outils sont partiellement visibles et certains se ressemblent beaucoup. Pour relever ces défis, nous proposons d'ajouter une caméra supplémentaire filmant la table opératoire. Notre objectif est donc de détecter la présence des outils dans les deux types de flux vidéo : les vidéos du microscope et les vidéos de la table opératoire. Le premier enregistre l'oeil du patient et le second enregistre les activités de la table opératoire. Deux tâches sont proposées pour détecter les outils dans les vidéos de la table : la détection des changements et la détection de présence d'outil. Dans un premier temps, nous proposons un système similaire pour ces deux tâches. Il est basé sur l’extraction des caractéristiques visuelles avec des méthodes de classification classique. Il fournit des résultats satisfaisants pour la détection de changement, cependant, il fonctionne insuffisamment bien pour la tâche de détection de présence des outils sur la table. Dans un second temps, afin de résoudre le problème du choix des caractéristiques, nous utilisons des architectures d’apprentissage profond pour la détection d'outils chirurgicaux sur les deux types de vidéo. Pour surmonter les défis rencontrés dans les vidéos de la table, nous proposons de générer des vidéos artificielles imitant la scène de la table opératoire et d’utiliser un réseau de neurones à convolutions (CNN) à base de patch. Enfin, nous exploitons l'information temporelle en utilisant un réseau de neurones récurrent analysant les résultats de CNNs. Contrairement à notre hypothèse, les expérimentations montrent des résultats insuffisants pour la détection de présence des outils sur la table, mais de très bons résultats dans les vidéos du microscope. Nous obtenons des résultats encore meilleurs dans les vidéos du microscope après avoir fusionné l’information issue de la détection des changements sur la table et la présence des outils dans l’oeil. / The digital era is increasingly changing the world due to the sheer volume of data produced every day. The medical domain is highly affected by this revolution, because analysing this data can be a source of education/support for the clinicians. In this thesis, we propose to reuse the surgery videos recorded in the operating rooms for computer-assisted surgery system. We are chiefly interested in recognizing the surgical gesture being performed at each instant in order to provide relevant information. To achieve this goal, this thesis addresses the surgical tool recognition problem, with applications in cataract surgery. The main objective of this thesis is to address the surgical tool recognition problem in cataract surgery videos.In the surgical field, those tools are partially visible in videos and highly similar to one another. To address the visual challenges in the cataract surgical field, we propose to add an additional camera filming the surgical tray. Our goal is to detect the tool presence in the two complementary types of videos: tool-tissue interaction and surgical tray videos. The former records the patient's eye and the latter records the surgical tray activities.Two tasks are proposed to perform the task on the surgical tray videos: tools change detection and tool presence detection.First, we establish a similar pipeline for both tasks. It is based on standard classification methods on top of visual learning features. It yields satisfactory results for the tools change task, howev-lateer, it badly performs the surgical tool presence task on the tray. Second, we design deep learning architectures for the surgical tool detection on both video types in order to address the difficulties in manually designing the visual features.To alleviate the inherent challenges on the surgical tray videos, we propose to generate simulated surgical tray scenes along with a patch-based convolutional neural network (CNN).Ultimately, we study the temporal information using RNN processing the CNN results. Contrary to our primary hypothesis, the experimental results show deficient results for surgical tool presence on the tray but very good results on the tool-tissue interaction videos. We achieve even better results in the surgical field after fusing the tool change information coming from the tray and tool presence signals on the tool-tissue interaction videos.