No description available.

The goal of Intelligent Tutoring systems (ITSs) is to engage the students in sustained reasoning activity and to interact with students based on a deep understanding of student behavior. In order to understand student behavior, ITSs rely on student modeling methods to observes student actions in the tutor and creates a quantitative representation of student knowledge, interests, affective states. Good student models are going to effectively help ITSs customize instructions, engage student's interest and then promote learning. Thus, the work of building ITSs and advancing student modeling should be considered as two interconnected components of one system rather than two separate topics. In this work, we utilized the theoretical support of a well-known learning science theory, the spacing effect, to guide the development of an ITS, called Automatic Reassessment and Relearning System (ARRS). ARRS not only validated the effectiveness of spacing effect, but it also served as a testing field which allowed us to find out new approaches to improve student learning by conducting large-scale randomized controlled trials (RCTs). The rich data set we gathered from ARRS has advanced our understanding of robust learning and helped us build student models with advanced data mining methods. At the end, we designed a set of API that supports the development of ARRS in next generation ASSISTments platform and adopted deep learning algorithms to further improve retention performance prediction. We believe our work is a successful example of combining theory and practice to advance science and address real- world problems.

Recurrent neural networks (RNNs) are state of the art sequential machine learning tools, but have difficulty learning sequences with long-range dependencies due to the exponential growth or decay of gradients backpropagated through the RNN. Some methods overcome this problem by modifying the standard RNN architecure to force the recurrent weight matrix W to remain orthogonal throughout training. The first half of this thesis presents a novel orthogonal RNN architecture that enforces orthogonality of W by parametrizing with a skew-symmetric matrix via the Cayley transform. We present rules for backpropagation through the Cayley transform, show how to deal with the Cayley transform's singularity, and compare its performance on benchmark tasks to other orthogonal RNN architectures. The second half explores two deep learning approaches to problems in RNA secondary structure inference and compares them to a standard structure inference tool, the nearest neighbor thermodynamic model (NNTM). The first uses RNNs to detect paired or unpaired nucleotides in the RNA structure, which are then converted into synthetic auxiliary data that direct NNTM structure predictions. The second method uses recurrent and convolutional networks to directly infer RNA base pairs. In many cases, these approaches improve over NNTM structure predictions by 20-30 percentage points.

Orientador: Aparecido Augusto de Carvalho / Abstract: In the last few years, several studies have been carried out showing that neuromuscular electrical stimulation (NMES) can produce good therapeutic results in patients with spinal cord injury (SCI). This research introduces a new robust and intelligent control-based methodology for human lower limb rehabilitation via NMES using a continuous-time control technique named robust integral of the sign of the error (RISE). Although in the literature the RISE controller has shown good results without any fine-tuning method, a trial and error approach would quickly lead to muscle fatigue in SCI patients. Therefore, it was shown in this study that the control performance for robustly tracking a reference signal can be improved through the proposed approach by providing an intelligent tuning for each voluntary. Simulation results with a mathematical model and eight identified subjects from the literature are provided, and real experiments are performed with seven healthy and two paraplegic subjects. Besides, this research introduces the application of deep and dynamic neural networks namely the multilayer perceptron, a simple recurrent neural network, and the Long Short-Term memory architecture, to identify the nonlinear and time-varying relationship between the supplied NMES and achieved angular position. Identification results indicate good fitting to data and very low mean square error using few data for training, proving to be very prospective methods for proposing control-oriented ... (Complete abstract click electronic access below) / Resumo: Nos últimos anos, vários estudos foram realizados mostrando que a estimulação elétrica neuromuscular (EENM) pode produzir bons resultados terapêuticos em pacientes com lesão medular (LM). Esta pesquisa introduz uma nova metodologia robusta e inteligente baseada em controle para a reabilitação de membros inferiores humanos via EENM usando uma técnica de controle de tempo contínuo chamada robust integral of the sign of the error (RISE). Embora na literatura o controlador RISE tem demonstrado bons resultados sem qualquer método de ajuste fino, uma abordagem de tentativa e erro poderia levar rapidamente à fadiga muscular em pacientes com LM. Portanto, foi mostrado nesse estudo que o desempenho do controle para rastrear com robustez um sinal de referência pode ser melhorado através da abordagem proposta, fornecendo um ajuste inteligente para cada voluntário. Resultados de simulação com um modelo matemático e oito sujeitos identificados da literatura são fornecidos, e experimentos reais são feitos com sete indivíduos saudáveis ​​e dois paraplégicos. Além disso, esta pesquisa introduz a aplicação de redes neurais profundas e dinâmicas, especificamente o perceptron multicamadas, uma rede neural recorrente simples e a arquitetura Long Short-Term Memory, para identificar a relação não-linear e variante no tempo entre a EENM fornecida e a posição angular alcançada. Os resultados de identificação indicam boa adaptação aos dados e erro quadrático médio muito baixo usando poucos dados para... (Resumo completo, clicar acesso eletrônico abaixo) / Mestre

Statistical shift-reduce parsing involves the interplay of representation learning, structured learning, and inexact search. This dissertation considers approaches that tightly integrate these three elements and explores three novel models for shift-reduce CCG parsing. First, I develop a dependency model, in which the selection of shift-reduce action sequences producing a dependency structure is treated as a hidden variable; the key components of the model are a dependency oracle and a learning algorithm that integrates the dependency oracle, the structured perceptron, and beam search. Second, I present expected F-measure training and show how to derive a globally normalized RNN model, in which beam search is naturally incorporated and used in conjunction with the objective to learn shift-reduce action sequences optimized for the final evaluation metric. Finally, I describe an LSTM model that is able to construct parser state representations incrementally by following the shift-reduce syntactic derivation process; I show expected F-measure training, which is agnostic to the underlying neural network, can be applied in this setting to obtain globally normalized greedy and beam-search LSTM shift-reduce parsers.

L’ère numérique change de plus en plus le monde en raison de la quantité de données récoltées chaque jour. Le domaine médical est fortement affecté par cette explosion, car l’exploitation de ces données est un véritable atout pour l’aide à la pratique médicale. Dans cette thèse, nous proposons d’utiliser les vidéos chirurgicales dans le but de créer un système de chirurgie assistée par ordinateur. Nous nous intéressons principalement à reconnaître les gestes chirurgicaux à chaque instant afin de fournir aux chirurgiens des recommandations et des informations pertinentes. Pour ce faire, l’objectif principal de cette thèse est de reconnaître les outils chirurgicaux dans les vidéos de chirurgie de la cataracte. Dans le flux vidéo du microscope, ces outils sont partiellement visibles et certains se ressemblent beaucoup. Pour relever ces défis, nous proposons d'ajouter une caméra supplémentaire filmant la table opératoire. Notre objectif est donc de détecter la présence des outils dans les deux types de flux vidéo : les vidéos du microscope et les vidéos de la table opératoire. Le premier enregistre l'oeil du patient et le second enregistre les activités de la table opératoire. Deux tâches sont proposées pour détecter les outils dans les vidéos de la table : la détection des changements et la détection de présence d'outil. Dans un premier temps, nous proposons un système similaire pour ces deux tâches. Il est basé sur l’extraction des caractéristiques visuelles avec des méthodes de classification classique. Il fournit des résultats satisfaisants pour la détection de changement, cependant, il fonctionne insuffisamment bien pour la tâche de détection de présence des outils sur la table. Dans un second temps, afin de résoudre le problème du choix des caractéristiques, nous utilisons des architectures d’apprentissage profond pour la détection d'outils chirurgicaux sur les deux types de vidéo. Pour surmonter les défis rencontrés dans les vidéos de la table, nous proposons de générer des vidéos artificielles imitant la scène de la table opératoire et d’utiliser un réseau de neurones à convolutions (CNN) à base de patch. Enfin, nous exploitons l'information temporelle en utilisant un réseau de neurones récurrent analysant les résultats de CNNs. Contrairement à notre hypothèse, les expérimentations montrent des résultats insuffisants pour la détection de présence des outils sur la table, mais de très bons résultats dans les vidéos du microscope. Nous obtenons des résultats encore meilleurs dans les vidéos du microscope après avoir fusionné l’information issue de la détection des changements sur la table et la présence des outils dans l’oeil. / The digital era is increasingly changing the world due to the sheer volume of data produced every day. The medical domain is highly affected by this revolution, because analysing this data can be a source of education/support for the clinicians. In this thesis, we propose to reuse the surgery videos recorded in the operating rooms for computer-assisted surgery system. We are chiefly interested in recognizing the surgical gesture being performed at each instant in order to provide relevant information. To achieve this goal, this thesis addresses the surgical tool recognition problem, with applications in cataract surgery. The main objective of this thesis is to address the surgical tool recognition problem in cataract surgery videos.In the surgical field, those tools are partially visible in videos and highly similar to one another. To address the visual challenges in the cataract surgical field, we propose to add an additional camera filming the surgical tray. Our goal is to detect the tool presence in the two complementary types of videos: tool-tissue interaction and surgical tray videos. The former records the patient's eye and the latter records the surgical tray activities.Two tasks are proposed to perform the task on the surgical tray videos: tools change detection and tool presence detection.First, we establish a similar pipeline for both tasks. It is based on standard classification methods on top of visual learning features. It yields satisfactory results for the tools change task, howev-lateer, it badly performs the surgical tool presence task on the tray. Second, we design deep learning architectures for the surgical tool detection on both video types in order to address the difficulties in manually designing the visual features.To alleviate the inherent challenges on the surgical tray videos, we propose to generate simulated surgical tray scenes along with a patch-based convolutional neural network (CNN).Ultimately, we study the temporal information using RNN processing the CNN results. Contrary to our primary hypothesis, the experimental results show deficient results for surgical tool presence on the tray but very good results on the tool-tissue interaction videos. We achieve even better results in the surgical field after fusing the tool change information coming from the tray and tool presence signals on the tool-tissue interaction videos.

Large collections of historical document images have been collected by companies and government institutions for decades. More recently, these collections have been made available to a larger public via the Internet. However, to make accessing them truly useful, the contained images need to be made readable and searchable. One step in that direction is document image binarization, the separation of text foreground from page background. This separation makes the text shown in the document images easier to process by humans and other image processing algorithms alike. While reasonably well working binarization algorithms exist, it is not sufficient to just being able to perform the separation of foreground and background well. This separation also has to be achieved in an efficient manner, in terms of execution time, but also in terms of training data used by machine learning based methods. This is necessary to make binarization not only theoretically possible, but also practically viable. In this thesis, we explore different ways to achieve efficient binarization in terms of execution time by improving the implementation and the algorithm of a state-of-the-art binarization method. We find that parameter prediction, as well as mapping the algorithm onto the graphics processing unit (GPU) help to improve its execution performance. Furthermore, we propose a binarization algorithm based on recurrent neural networks and evaluate the choice of its design parameters with respect to their impact on execution time and binarization quality. Here, we identify a trade-off between binarization quality and execution performance based on the algorithm’s footprint size and show that dynamically weighted training loss tends to improve the binarization quality. Lastly, we address the problem of training data efficiency by evaluating the use of interactive machine learning for reducing the required amount of training data for our recurrent neural network based method. We show that user feedback can help to achieve better binarization quality with less training data and that visualized uncertainty helps to guide users to give more relevant feedback. / Scalable resource-efficient systems for big data analytics

Natural Language Processing (NLP) is a field studying computer processing of human language. Recently, neural network language models, a subset of machine learning, have been used to great effect in this field. However, research remains focused on the English language, with few implementations in other languages of the world. This work focuses on how NLP techniques can be used for the task of grammar and spelling correction in the Swedish language, in order to investigate how language models can be applied to non-English languages. We use a controlled experiment to find the hyperparameters most suitable for grammar and spelling correction on the Göteborgs-Posten corpus, using a Long Short-term Memory Recurrent Neural Network. We present promising results for Swedish-specific grammar correction tasks using this kind of neural network; specifically, our network has a high accuracy in completing these tasks, though the accuracy achieved for language-independent typos remains low.

Submitted by Renata Lopes (renatasil82@gmail.com) on 2017-04-24T17:21:05Z No. of bitstreams: 1 leandrorodriguesmansosilva.pdf: 691785 bytes, checksum: 4024e0e319f1469cc354c2c346a90dbe (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2017-04-24T17:59:43Z (GMT) No. of bitstreams: 1 leandrorodriguesmansosilva.pdf: 691785 bytes, checksum: 4024e0e319f1469cc354c2c346a90dbe (MD5) / Made available in DSpace on 2017-04-24T17:59:43Z (GMT). No. of bitstreams: 1 leandrorodriguesmansosilva.pdf: 691785 bytes, checksum: 4024e0e319f1469cc354c2c346a90dbe (MD5) Previous issue date: 2012-02-29 / CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / A distorção harmônica, dentre outras formas de poluição na rede de sistemas de energia, é um importante problema para as concessionárias. De fato, o aumento do uso de dispositivos não-lineares na indústria resultou em um aumento direto da distorção harmônica nos sistemas elétricos de potência nos últimos anos. Com isso, a modelagem destas cargas e suas interações se tornaram de grande importância, e portanto, o uso de novas técnicas computacionais passou a ser de grande interesse para este fim. Neste contexto, este trabalho descreve uma metodologia baseada em técnicas de Inteligência Computacional (Redes Neurais Artificiais (RNA)s e Lógica Fuzzy (LF)), proposta para modelagem de cargas não-lineares presentes em sistemas elétricos de potência, bem como a estimação de sua parcela na distorção harmônica do sistema. A principal vantagem deste método é que apenas as formas de onda de tensão e corrente no ponto de acoplamento comum precisam ser medidas, além disso esta técnica pode ser aplicada na modelagem de cargas monofásicas bem como cargas trifásicas. / The harmonic distortin, among other forms of pollution to the electric power systems is an important issue for electric utilities. In fact, the increased use of nonlinear devices in industry has resulted in direct increase of harmonic distortion in industrial power grids in recent years. Thus, the modeling of these loads and the understanding of their interactions with the system have became of great importance, then the use of computational-based techniques has emerged as a suitable tool to deal with these requirements. In this context, this work describes a methodology based on Computational Intelligence (Artificial Neural Networks (ANN)s and Fuzzy Logic (FL)) for modeling nonlinear loads present in electric power systems, as well as the estimation of their contribution in the harmonic distortion. The main advantage of this technique is that only the waveforms of voltages and currents at the point of common coupling must be measured and it can be applied to model single and three phase loads.

Cette thèse porte sur l'obtention rigoureuse de limites de champ moyen pour la dynamique continue de grands réseaux de neurones hétérogènes. Nous considérons des neurones à taux de décharge, et sujets à un bruit Brownien additif. Le réseau est entièrement connecté, avec des poids de connections dont la variance décroît comme l'inverse du nombre de neurones conservant un effet non trivial dans la limite thermodynamique. Un second type d'hétérogénéité, interprété comme une position spatiale, est considéré au niveau de chaque cellule. Pour la pertinence biologique, nos modèles incluent ou bien des délais, ainsi que des moyennes et variances de connections, dépendants de la distance entre les cellules, ou bien des synapses dépendantes de l'état des deux neurones post- et présynaptique. Ce dernier cas s'applique au modèle de Kuramoto pour les oscillateurs couplés. Quand les poids synaptiques sont Gaussiens et indépendants, nous prouvons un principe de grandes déviations pour la mesure empirique de l'état des neurones. La bonne fonction de taux associée atteint son minimum en une unique mesure de probabilité, impliquant convergence et propagation du chaos sous la loi "averaged". Dans certains cas, des résultats "quenched" sont obtenus. La limite est solution d'une équation implicite, non Markovienne, dans laquelle le terme d'interactions est remplacé par un processus Gaussien qui dépend de la loi de la solution du réseau entier. Une universalité de cette limite est prouvée, dans le cas de poids synaptiques non-Gaussiens avec queues sous-Gaussiennes. Enfin, quelques résultats numérique sur les réseau aléatoires sont présentés, et des perspectives discutées. / This thesis addresses the rigorous derivation of mean-field results for the continuous time dynamics of heterogeneous large neural networks. In our models, we consider firing-rate neurons subject to additive noise. The network is fully connected, with highly random connectivity weights. Their variance scales as the inverse of the network size, and thus conserves a non-trivial role in the thermodynamic limit. Moreover, another heterogeneity is considered at the level of each neuron. It is interpreted as a spatial location. For biological relevance, a model considered includes delays, mean and variance of connections depending on the distance between cells. A second model considers interactions depending on the states of both neurons at play. This last case notably applies to Kuramoto's model of coupled oscillators. When the weights are independent Gaussian random variables, we show that the empirical measure of the neurons' states satisfies a large deviations principle, with a good rate function achieving its minimum at a unique probability measure, implying averaged convergence of the empirical measure and propagation of chaos. In certain cases, we also obtained quenched results. The limit is characterized through a complex non Markovian implicit equation in which the network interaction term is replaced by a non-local Gaussian process whose statistics depend on the solution over the whole neural field. We further demonstrate the universality of this limit, in the sense that neuronal networks with non-Gaussian interconnections but sub-Gaussian tails converge towards it. Moreover, we present a few numerical applications, and discuss possible perspectives.