Dois problemas de planejamento de trajetórias são tratados nesta dissertação, sendo um discreto e outro contínuo. O problema discreto consiste em estabelecer todos os estados intermediários de uma trajetória para levar um conjunto de quatro blocos de uma posição inicial à uma posição meta. O problema contínuo consiste em planejar e controlar a trajetória do braço mecânico PUMA 560. A classe de modelos que se utilizou nesta dissertação foram os modelos parcialmente recorrentes. O problema discreto foi utilizado com a finalidade de comparar os seis modelos propostos, buscando obter um modelo com bom desempenho para resolução de problemas de produção de seqüências temporais. Para o problema contínuo aplicou-se apenas o modelo que apresentou melhor desempenho na resolução do problema discreto. Em ambos os casos são apresentados como entrada para a rede, o ponto inicial e o ponto meta. Dois tipos de testes foram aplicados as arquiteturas: teste de produção e de generalização de seqüências temporais. Para cada problema foram criados quatro tipos distintos de trajetórias, com graus de complexidades diferentes. Para o problema discreto, em média, a arquitetura com realimentação da camada de saída para a camada de entrada e da camada de entrada para ela mesma, todos-para-todos, foi a que apresentou menor número de épocas e também os menores valores de erro durante o treinamento. Foi o único que conseguiu recuperar todos os padrões treinados e de forma geral apresentou melhor capacidade de generalização. Por isto, este modelo foi escolhido para ser aplicado na resolução do problema contínuo, tendo bom desempenho, conseguindo reproduzir as trajetórias treinadas com grande precisão. Para o problema discreto todos os modelos apresentaram baixa capacidade de generalização. Para o problema contínuo o modelo abordado apresentou-se de forma satisfatória mediante o acréscimo de ruído. / Two trajectory planning problems are discussed in this work, one of them being discrete and the other continuous. The discrete problem consists in establishing all the intermediate states o f a trajectory to move a set of four blocks from a initial to a goal position. The continuous problem consists in planning and controlling the trajectory of the PUMA 560 mechanical arm. The class of models utilized in this work were the partially recurrent models. The discrete problem was used in order to compare the six proposed models, aiming at the acquisition of a model with a good performance for the resolution of production of temporal sequence problems. For the continuous problem, only the model that presented better performance in solving the discrete problem was applied. The initial and goal point are presented as input for the network in both problems. Two types of tests were applied to the architectures: production and generalization of temporal sequence tests. Four distinct types of trajectories with different complexity levels were created for each problem. In average, for the discrete problem, the architecture with feedback from the output to the input layer and from input layer to itself all-to-all presented the lowest epoch number in addition to the lowest error values during the training. This was the only model that managed to recover all the patterns trained and in general presented better generalization capacity. For this reason, this model was chosen to be applied in the resolution of the continuous problem. It presented a good performance to the production of mechanical arm trajectories, managing to reproduce the trained trajectories with great accuracy. For the discrete problem, all the models presented low generalization capacity. For the continuous problem, the approached model presented itself in a satisfactmy manner by means of noise addition.

<p>Recurrent neural networks have been used for time-series prediction with good results. In this dissertation recurrent neural networks are compared with time-delayed feed forward networks, feed forward networks and linear regression models on a prediction task. The data used in all experiments is real-world sales data containing two kinds of segments: campaign segments and non-campaign segments. The task is to make predictions of sales under campaigns. It is evaluated if more accurate predictions can be made when only using the campaign segments of the data.</p><p>Throughout the entire project a knowledge discovery process, identified in the literature has been used to give a structured work-process. The results show that the recurrent network is not better than the other evaluated algorithms, in fact, the time-delayed feed forward neural network showed to give the best predictions. The results also show that more accurate predictions could be made when only using information from campaign segments.</p>

Σήμερα η μελέτη των τεχνητών νευρωνικών δικτύων είναι ένα ώριμο επιστημονικό πεδίο. Τα πρώτα μοντέλα νευρωνικών δικτύων έκαναν την εμφάνιση τους την δεκαετία 1940 έως 1950, ξεκινώντας από το βασικό μοντέλο του νευρώνα του Mc Culloch-Pitls και τον πρώτο αλγόριθμο εκπαίδευσης ενός νευρώνα, τον γνωστό Perceptron του Frank Rosenblatt. Σήμερα υπάρχουν πληθώρα νευρωνικών μοντέλων που ακολουθούν διάφορα πρότυπα μάθησης όπως εκπαίδευση με εποπτεία (επίβλεψη) εκπαίδευση χωρίς εποπτεία κ.α. Η εργασία αυτή αποτελείται από 6 κεφάλαια ξεκινώντας από τις βασικές έννοιες των τεχνητών νευρωνικών δικτύων και συνεχίζοντας μέχρι την ανάλυση των ανατροφοδοτούμενων νευρωνικών δικτύων καθώς και την χρήση τους στον έλεγχο συστημάτων παρουσιάζοντας και διάφορες εφαρμογές τους. Στο πρώτο εισαγωγικό κεφάλαιο αναφέρουμε τις βασικές αρχές των τεχνητών νευρωνικών δικτύων και την αντιστοιχία τους με τον φυσικό νευρώνα του ανθρώπου. Παραθέτουμε επίσης μια σύντομη ιστορική αναδρομή Στην συνέχεια στο κεφάλαιο 2 ασχολούμαστε με τα ανατροφοδοτούμενα νευρωνικά δίκτυα. Δίνεται ένας ορισμός τον ανατροφοδοτούμενων νευρωνικών δικτύων (recurrent neural networks RNN) και αναφέρονται τα κυριότερα και δημοφιλέστερα είδη αυτών. Δίνοντας μια σύντομη ανάλυση της λειτουργίας τους. Το τρίτο κεφάλαιο ασχολείται με την εκπαίδευση των νευρωνικών δικτύων και τους διάφορους αλγόριθμους εκπαίδευσης. Ξεκινώντας από τον αλγόριθμο εκπαίδευσης του πιο απλού νευρωνικού δικτύου του Perceptron και καταλήγοντας στον αλγόριθμο Back-Propagation. Το τέταρτο κεφάλαιο αναφέρεται στον έλεγχο συστημάτων και την χρήση των νευρωνικών και ανατροφοδοτούμενων νευρωνικών δικτύων σε αυτόν. Αναλύονται οι διάφορες μοντελοποιήσεις καθώς και οι δομές ελέγχου (υπό επίβλεψη , αντίστροφος έλεγχος, προσαρμοστικός γραμμικός έλεγχος κ.α.) Στα δύο τελευταία κεφάλαια παραθέτουμε ένα παράδειγμα χρήσης ενός απλού ανατροφοδοτούμενου νευρωνικού δικτύου (simple recurrent network SRN) στον έλεγχο και τέλος εφαρμογές των νευρωνικών δικτύων σε διάφορους τομείς. / -

In this thesis, we explore the interface between symbolic and dynamical system computation, with particular regard to dynamical system models of neuronal networks. In doing so, we adhere to a definition of computation as the physical realization of a formal system, where we say that a dynamical system performs a computation if a correspondence can be found between its dynamics on a vectorial space and the formal system’s dynamics on a symbolic space. Guided by this definition, we characterize computation in a range of neuronal network models. We first present a constructive mapping between a range of formal systems and Recurrent Neural Networks (RNNs), through the introduction of a Versatile Shift and a modular network architecture supporting its real-time simulation. We then move on to more detailed models of neural dynamics, characterizing the computation performed by networks of delay-pulse-coupled oscillators supporting the emergence of heteroclinic dynamics. We show that a correspondence can be found between these networks and Finite-State Transducers, and use the derived abstraction to investigate how noise affects computation in this class of systems, unveiling a surprising facilitatory effect on information transmission. Finally, we present a new dynamical framework for computation in neuronal networks based on the slow-fast dynamics paradigm, and discuss the consequences of our results for future work, specifically for what concerns the fields of interactive computation and Artificial Intelligence.

Dois problemas de planejamento de trajetórias são tratados nesta dissertação, sendo um discreto e outro contínuo. O problema discreto consiste em estabelecer todos os estados intermediários de uma trajetória para levar um conjunto de quatro blocos de uma posição inicial à uma posição meta. O problema contínuo consiste em planejar e controlar a trajetória do braço mecânico PUMA 560. A classe de modelos que se utilizou nesta dissertação foram os modelos parcialmente recorrentes. O problema discreto foi utilizado com a finalidade de comparar os seis modelos propostos, buscando obter um modelo com bom desempenho para resolução de problemas de produção de seqüências temporais. Para o problema contínuo aplicou-se apenas o modelo que apresentou melhor desempenho na resolução do problema discreto. Em ambos os casos são apresentados como entrada para a rede, o ponto inicial e o ponto meta. Dois tipos de testes foram aplicados as arquiteturas: teste de produção e de generalização de seqüências temporais. Para cada problema foram criados quatro tipos distintos de trajetórias, com graus de complexidades diferentes. Para o problema discreto, em média, a arquitetura com realimentação da camada de saída para a camada de entrada e da camada de entrada para ela mesma, todos-para-todos, foi a que apresentou menor número de épocas e também os menores valores de erro durante o treinamento. Foi o único que conseguiu recuperar todos os padrões treinados e de forma geral apresentou melhor capacidade de generalização. Por isto, este modelo foi escolhido para ser aplicado na resolução do problema contínuo, tendo bom desempenho, conseguindo reproduzir as trajetórias treinadas com grande precisão. Para o problema discreto todos os modelos apresentaram baixa capacidade de generalização. Para o problema contínuo o modelo abordado apresentou-se de forma satisfatória mediante o acréscimo de ruído. / Two trajectory planning problems are discussed in this work, one of them being discrete and the other continuous. The discrete problem consists in establishing all the intermediate states o f a trajectory to move a set of four blocks from a initial to a goal position. The continuous problem consists in planning and controlling the trajectory of the PUMA 560 mechanical arm. The class of models utilized in this work were the partially recurrent models. The discrete problem was used in order to compare the six proposed models, aiming at the acquisition of a model with a good performance for the resolution of production of temporal sequence problems. For the continuous problem, only the model that presented better performance in solving the discrete problem was applied. The initial and goal point are presented as input for the network in both problems. Two types of tests were applied to the architectures: production and generalization of temporal sequence tests. Four distinct types of trajectories with different complexity levels were created for each problem. In average, for the discrete problem, the architecture with feedback from the output to the input layer and from input layer to itself all-to-all presented the lowest epoch number in addition to the lowest error values during the training. This was the only model that managed to recover all the patterns trained and in general presented better generalization capacity. For this reason, this model was chosen to be applied in the resolution of the continuous problem. It presented a good performance to the production of mechanical arm trajectories, managing to reproduce the trained trajectories with great accuracy. For the discrete problem, all the models presented low generalization capacity. For the continuous problem, the approached model presented itself in a satisfactmy manner by means of noise addition.

Recurrent neural networks have been used for time-series prediction with good results. In this dissertation recurrent neural networks are compared with time-delayed feed forward networks, feed forward networks and linear regression models on a prediction task. The data used in all experiments is real-world sales data containing two kinds of segments: campaign segments and non-campaign segments. The task is to make predictions of sales under campaigns. It is evaluated if more accurate predictions can be made when only using the campaign segments of the data. Throughout the entire project a knowledge discovery process, identified in the literature has been used to give a structured work-process. The results show that the recurrent network is not better than the other evaluated algorithms, in fact, the time-delayed feed forward neural network showed to give the best predictions. The results also show that more accurate predictions could be made when only using information from campaign segments.

Semantic labeling is becoming more and more popular among researchers in computer vision and machine learning. Many applications, such as autonomous driving, tracking, indoor navigation, augmented reality systems, semantic searching, medical imaging are on the rise, requiring more accurate and efficient segmentation mechanisms. In recent years, deep learning approaches based on Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have dramatically emerged as the dominant paradigm for solving many problems in computer vision and machine learning. The main focus of this thesis is to investigate robust approaches that can tackle the challenging semantic labeling tasks including semantic instance segmentation and scene understanding. In the first approach, we convert the classic variational Level Set method to a learnable deep framework by proposing a novel definition of contour evolution named Recurrent Level Set (RLS). The proposed RLS employs Gated Recurrent Units to solve the energy minimization of a variational Level Set functional. The curve deformation processes in RLS is formulated as a hidden state evolution procedure and is updated by minimizing an energy functional composed of fitting forces and contour length. We show that by sharing the convolutional features in a fully end-to-end trainable framework, RLS is able to be extended to Contextual Recurrent Level Set (CRLS) Networks to address semantic segmentation in the wild problem. The experimental results have shown that our proposed RLS improves both computational time and segmentation accuracy against the classic variational Level Set-based methods whereas the fully end-to-end system CRLS achieves competitive performance compared to the state-of-the-art semantic segmentation approaches on PAS CAL VOC 2012 and MS COCO 2014 databases. The second proposed approach, Contextual Recurrent Residual Networks (CRRN), inherits all the merits of sequence learning information and residual learning in order to simultaneously model long-range contextual infor- mation and learn powerful visual representation within a single deep network. Our proposed CRRN deep network consists of three parts corresponding to sequential input data, sequential output data and hidden state as in a recurrent network. Each unit in hidden state is designed as a combination of two components: a context-based component via sequence learning and a visualbased component via residual learning. That means, each hidden unit in our proposed CRRN simultaneously (1) learns long-range contextual dependencies via a context-based component. The relationship between the current unit and the previous units is performed as sequential information under an undirected cyclic graph (UCG) and (2) provides powerful encoded visual representation via residual component which contains blocks of convolution and/or batch normalization layers equipped with an identity skip connection. Furthermore, unlike previous scene labeling approaches [1, 2, 3], our method is not only able to exploit the long-range context and visual representation but also formed under a fully-end-to-end trainable system that effectively leads to the optimal model. In contrast to other existing deep learning networks which are based on pretrained models, our fully-end-to-end CRRN is completely trained from scratch. The experiments are conducted on four challenging scene labeling datasets, i.e. SiftFlow, CamVid, Stanford background, and SUN datasets, and compared against various state-of-the-art scene labeling methods.

This thesis proposes the use of Machine Learning (ML) to two very distinct, yet compelling, applications – predicting cardiac arrhythmia episodes and predicting base station association in 5G networks comprising of virtual cells. In the first scenario, Support Vector Machines (SVMs) are used to classify features extracted from electrocardiogram (EKG) signals. The second problem requires a different formulation departing from traditional ML classification where the objective is to partition feature space into constituent class regions. Instead, the intention here is to identify temporal patterns in unequal-length sequences. Using Recurrent Neural Networks (RNNs), it is demonstrated that accurate predictions can be made as to the base station most likely to provide connectivity for a mobile device as it moves. Atrial Fibrillation (AF) is a common cardiac arrhythmia affecting several million people in the United States. It is a condition in which the upper chambers of the heart are unable to contract effectively leading to inhibited blood flow to the ventricles. The stagnation of blood is one of the major risk factors for stroke. The Computers in Cardiology Challenge 2001 was organized to further research into the prediction of episodes of AF. This research revisits the problem with some modifications. Patient-specific classifiers are developed for AF prediction using a different dataset and employing shorter EKG signal epochs. SVM classification yielded an average accuracy of just above 95% in identifying EKG epochs appearing just prior to fibrillatory rhythms. 5G cellular networks were envisaged to provide enhanced data rates for mobile broadband, support low-latency communication, and enable the Internet of Things (IoT). Handovers contribute to latency as mobile devices are switched between base stations due to movements. Given that customers may not be willing to continuously share their exact locations due to privacy concerns and the establishment of a mobile network architecture with dynamically created virtual cells, this research presents a solution for proactive mobility management using RNNs. A RNN is trained to identify patterns in variable-length sequences of Received Signal Strength (RSS) values, where a mobile device is permitted to connect to more than a single base station at a time. A classification accuracy of over 98% was achieved in a simulation model that was set up emulating an urban environment.

No description available.

Recurrent Neural Networks (RNNs) are variants of Neural Networks that are able to learn temporal relationships between sequences presented to the neural network. RNNs are often employed to learn underlying relationships in time series and sequential data. This dissertation examines the extent of RNN’s memory retention and how it is influenced by different activation functions, network structures and recurrent network types. To investigate memory retention, three approaches (and variants thereof) are used. First the number of patterns each network is able to retain is measured. Thereafter the length of retention is investigated. Lastly the previous experiments are combined to measure the retention of patterns over time. During each investigation, the effect of using different activation functions and network structures are considered to determine the configurations’ effect on memory retention. The dissertation concludes that memory retention of a network is not necessarily improved when adding more parameters to a network. Activation functions have a large effect on the performance of RNNs when retaining patterns, especially temporal patterns. Deeper network structures have the trade-off of less memory retention per parameter in favour of the ability to model more complex relationships. / Dissertation (MSc (Computer Science))--University of Pretoria, 2020. / Computer Science / MSc (Computer Science) / Unrestricted