This thesis is about the study of relationships between experimental dynamical systems. The basic approach is to fit radial basis function maps between time delay embeddings of manifolds. We have shown that under certain conditions these maps are generically diffeomorphisms, and can be analysed to determine whether or not the manifolds in question are diffeomorphically related to each other. If not, a study of the distribution of errors may provide information about the lack of equivalence between the two. The method has applications wherever two or more sensors are used to measure a single system, or where a single sensor can respond on more than one time scale: their respective time series can be tested to determine whether or not they are coupled, and to what degree. One application which we have explored is the determination of a minimum embedding dimension for dynamical system reconstruction. In this special case the diffeomorphism in question is closely related to the predictor for the time series itself. Linear transformations of delay embedded manifolds can also be shown to have nonlinear inverses under the right conditions, and we have used radial basis functions to approximate these inverse maps in a variety of contexts. This method is particularly useful when the linear transformation corresponds to the delay embedding of a finite impulse response filtered time series. One application of fitting an inverse to this linear map is the detection of periodic orbits in chaotic attractors, using suitably tuned filters. This method has also been used to separate signals with known bandwidths from deterministic noise, by tuning a filter to stop the signal and then recovering the chaos with the nonlinear inverse. The method may have applications to the cancellation of noise generated by mechanical or electrical systems. In the course of this research a sophisticated piece of software has been developed. The program allows the construction of a hierarchy of delay embeddings from scalar and multi-valued time series. The embedded objects can be analysed graphically, and radial basis function maps can be fitted between them asynchronously, in parallel, on a multi-processor machine. In addition to a graphical user interface, the program can be driven by a batch mode command language, incorporating the concept of parallel and sequential instruction groups and enabling complex sequences of experiments to be performed in parallel in a resource-efficient manner.

Understanding the reasoning behind the low enrollment and retention rates of Underrepresented Minority (URM) students (African Americans, Hispanic Americans, and Native Americans) in the disciplines of science, technology, engineering, and mathematics (STEM) has concerned many researchers for decades. Numerous studies have used traditional statistical methods to identify factors that affect and predict student retention. Recently, researchers have relied on using data mining techniques for modeling student retention in higher education [1]. This research has used neural networks for performance modeling in order to obtain an adequate understanding of factors related to first year academic success and retention of URM at Virginia Commonwealth University. This research used feed forward back-propagation architecture for modeling. The student retention model was developed based on fall to fall retention in STEM majors. The overall freshman year GPA was used to model student academic success. Each model was built in two different ways: the first was built using all available student inputs, and the second using an optimized subset of student inputs. The optimized subset of the most relevant features that comes with the student, such as demographic attributes, high school rank, and SAT test scores was formed using genetic algorithms. A further step towards understanding the retention of URM groups in STEM fields was taken by conducting a series of focus groups with participants of an intervention program at VCU. Focus groups were designed to elicit responses from participants for identifying factors that affect their retention the most and provide more knowledge about their first year experiences, academically and socially. Results of the genetic algorithm and focus groups were incorporated into building a hybrid model using the most relevant student inputs. The developed hybrid model is shown to be a valuable tool in analyzing and predicting student academic success and retention. In particular, we have shown that identifying the most relevant student inputs from the student’s perspective can be incorporated with quantitative methodologies to build a tool that can be used and interpreted effectively by people who are related to the field of STEM retention and education. Further, the hybrid model performed comparable to the model developed using the optimized set of inputs that resulted from the genetic algorithm. The GPA prediction hybrid model was tested to determine how well it would predict the GPA for all students, majority students and URM students. The root mean squared error (RMSE) on a 4.0 scale was 0.45 for all students, 0.47 for majority students, and 0.45 for URM students. The hybrid retention model was able to predict student retention correctly for 74% of all students, 79% of majority students and 60% of URM students. The hybrid model’s accuracy was increased 3% compared to the model which used the optimized set of inputs.

This study aimed to develop a real time multiple fundamental frequency detection algorithm for real time pitch to MIDI conversion applications. The algorithm described here uses neural network classifiers to make classifications in order to define a chord pattern (combination of multiple fundamental frequencies). The first classification uses a binary decision tree that determines the root note (first note) in a combination of notes; this is achieved through a neural network binary classifier. For each leaf of the binary tree, each classifier determines the frequency group of the root note (low or high frequency) until only two frequencies are left to choose from. The second classifier determines the amount of polyphony, or number of notes played. This classifier is designed in the same fashion as the first, using a binary tree made up of neural network classifiers. The third classifier classifies the chord pattern that has been played. The chord classifier is chosen based on the root note and amount of polyphony, the first two classifiers constrain the third classifier to chords containing only a specific root not and a set polyphony. This allows for the classifier to be more focused and of a higher accuracy. To further increase accuracy, an error correction scheme was devised based on repetitive coding, a technique that holds out multiple frames and compares them in order to detect and correct errors. Repetitive coding significantly increases the classifiers accuracy; it was found that holding out three frames was suitable for real-time operation in terms of throughput, though holding out more frames further increases accuracy it was not suitable real time operation. The algorithm was tested on a common embedded platform, which through benchmarking showed the algorithm was well suited for real time operation.

No description available.

No description available.

In this study the prediction capabilities of Artificial Neural Networks and typical econometric methods are compared. This is done in the domains of Finance and Economics. Initially, the Neural Networks are shown to outperform traditional econometric models in forecasting nonlinear behaviour. The comparison is extended to indicate that the accuracy of share price forecasting is not necessarily improved when applying Neural Networks rather than traditional time series analysis. Finally, Neural Networks are used to forecast the South African inflation rates, and its performance is compared to that of vector error correcting models, which apparently outperform Artificial Neural Networks. / Prof. D.J. Marais

Digital steganography is gaining wide acceptance in the world of electronic copyright stamping. Digital media that are easy to steal, such as graphics, photos and audio files, are being tagged with both visible and invisible copyright stamp known as a digital watermark. However, these same methodologies are also used to hide communications between actors in criminal or covert activities. An inherent difficulty in developing steganography attacks is overcoming the variety of methods for hiding a message and the multitude of choices of available media. The steganalyst cannot create an attack until the hidden content method appears. When a message is visually transmitted in a non-textual format (i.e., in an image) it is referred to as a semagram. Semagrams are a subset of steganography and are relatively easy to create. However, detecting a hidden message in an image-based semagram is more difficult than detecting digital modifications to an image's structure. The trend in steganography is a decrease in detectable digital traces, and a move toward semagrams. This research outlines the creation of a novel, computer-based application, designed to detect the likely presence of a Morse Code based semagram message in an image. This application capitalizes on the adaptability and learning capabilities of various artificial neural network (NN) architectures, most notably hierarchical architectures. Four NN architectures [feed-forward Back-Propagation NN (BPNN), Self organizing Map (SOM), Neural Abstraction Pyramid (NAP), and a Hybrid Custom Network (HCN)] were tested for applicability to this domain with the best performing one being the HCN. Each NN was given a baseline set of training images (quantity based on NN architecture) then test images were presented, (each test set having 3,337 images). There were 36 levels of testing. Each subsequent test set representing an increase in complexity over the previous one. In the end, the HCN proved to be the NN of choice from among the four tested. The final HCN implementation was the only network able to successfully perform against all 36 levels. Additionally, the HCN, while only being trained on the base Morse Code images, successfully detected images in the 9 test sets of Morse Code isomorphs.

As the real estate market plays a central role in a countries financial situation, as a life insurer, a bank and a property developer, Skandia wants a method for better assessing the risks connected to the real estate market. The goal of this paper is to increase the understanding of property market risk and its covariate risks and to conduct an analysis of how a fall in real estate prices could affect Skandia’s exposed assets.This paper explores a recurrent neural network model with the aim of quantifying identified risk factors using exogenous data. The recurrent neural network model is compared to a vector autoregressive model with exogenous inputs that represent economic conditions.The results of this paper are inconclusive as to which method that produces the most accurate model under the specified settings. The recurrent neural network approach produces what seem to be better results in out-of-sample validation but both the recurrent neural network model and the vector autoregressive model fail to capture the hypothesized relationship between the exogenous and modeled variables. However producing results that does not fit previous assumptions, further research into artificial neural networks and tests with additional variables and longer sample series for calibration is suggested as the model preconditions are promising.

A growing body of research indicates that much of the brain’s computation is invisible from the activity of individual neurons, but instead instantiated via population-level dynamics. According to this ‘dynamical systems hypothesis’, population-level neural activity evolves according to underlying dynamics that are shaped by network connectivity. While these dynamics are not directly observable in empirical data, they can be inferred by studying the structure of population trajectories. Quantification of this structure, the ‘trajectory geometry’, can then guide thinking on the underlying computation. Alternatively, modeling neural populations as dynamical systems can predict trajectory geometries appropriate for particular tasks. This approach of characterizing and interpreting trajectory geometry is providing new insights in many cortical areas, including regions involved in motor control and areas that mediate cognitive processes such as decision-making. In this thesis, I advance the characterization of population structure by introducing hypothesis-guided metrics for the quantification of trajectory geometry. These metrics, trajectory tangling in primary motor cortex and trajectory divergence in the Supplementary Motor Area, abstract away from task-specific solutions and toward underlying computations and network constraints that drive trajectory geometry. Primate motor cortex (M1) projects to spinal interneurons and motoneurons, suggesting that motor cortex activity may be dominated by muscle-like commands. Observations during reaching lend support to this view, but evidence remains ambiguous and much debated. To provide a different perspective, we employed a novel behavioral paradigm that facilitates comparison between time-evolving neural and muscle activity. We found that single motor cortex neurons displayed many muscle-like properties, but the structure of population activity was not muscle-like. Unlike muscle activity, neural activity was structured to avoid ‘trajectory tangling’: moments where similar activity patterns led to dissimilar future patterns. Avoidance of trajectory tangling was present across tasks and species. Network models revealed a potential reason for this consistent feature: low trajectory tangling confers noise robustness. We were able to predict motor cortex activity from muscle activity by leveraging the hypothesis that muscle-like commands are embedded in additional structure that yields low trajectory tangling. The Supplementary Motor Area (SMA) has been implicated in many higher-order aspects of motor control. Previous studies have demonstrated that SMA might track motor context. We propose that this computation necessitates that neural activity avoids ‘trajectory divergence’: moments where two similar neural states become dissimilar in the future. Indeed, we found that population activity in SMA, but not in M1, reliably avoided trajectory divergence, resulting in fundamentally different geometries: cyclical in M1 and helix-like in SMA. Analogous structure emerged in artificial networks trained without versus with context-related inputs. These findings reveal that the geometries of population activity in SMA and M1 are fundamentally different, with direct implications regarding what computations can be performed by each area. The characterization and statistical analysis of trajectory geometry promises to advance our understanding of neural network function by providing interpretable, cohesive explanations for observed population structure. Commonality between individuals and networks can be uncovered and more generic, task-invariant, fundamental aspects of neural response can be explored.

University of Macau / Faculty of Science and Technology. / Department of Computer and Information Science