The research presented in this thesis examines the area of control systems for robots or animats (animal-like robots). Existing systems have problems in that they require a great deal of manual design or are limited to performing jobs of a single type. For these reasons, a better solution is desired. The system studied here is an Artificial Nervous System (ANS) which is biologically inspired; it is arranged as a hierarchy of layers containing modules operating in parallel. The ANS model has been developed to be flexible, scalable, extensible and modular. The ANS can be implemented using any suitable technology, for many different environments. The implementation focused on the two lowest layers (the reflex and action layers) of the ANS, which are concerned with control and rhythmic movement. Both layers were realised as Artificial Neural Networks (ANN) which were created using Evolutionary Algorithms (EAs). The task of the reflex layer was to control the position of an actuator (such as linear actuators or D.C. motors). The action layer performed the task of Central Pattern Generators (CPG), which produce rhythmic patterns of activity. In particular, different biped and quadruped gait patterns were created. An original neural model was specifically developed for assisting in the creation of these time-based patterns. It is shown in the thesis that Artificial Reflexes and CPGs can be configured successfully using this technique. The Artificial Reflexes were better at generalising across different actuators, without changes, than traditional controllers. Gaits such as pace, trot, gallop and pronk were successfully created using the CPGs. Experiments were conducted to determine whether modularity in the networks had an impact. It has been demonstrated that the degree of modularization in the network influences its evolvability, with more modular networks evolving more efficiently.

The marketing literature so far only considers attraction models with strict functional forms. Greater exibility can be achieved by the neural net based approach introduced which assesses brands' attraction values by means of a perceptron with one hidden layer. Using log-ratio transformed market shares as dependent variables stochastic gradient descent followed by a quasi-Newton method estimates parameters. For store-level data the neural net model performs better and implies a price response qualitatively different from the well-known MNL attraction model. Price elasticities of these competing models also lead to specific managerial implications. (author's abstract) / Series: Working Papers SFB "Adaptive Information Systems and Modelling in Economics and Management Science"

Microwave detection techniques based on the theory of perturbation of cavity resonators are commonly used to measure the permittivity and permeability of objects of dielectric and ferrite materials at microwave frequencies. When a small object is introduced into a microwave cavity resonator, the resonant frequency is perturbed. Since it is possible to measure the change in frequency with high accuracy, this provides a valuable method for measuring the electric and magnetic properties of the object. Likewise, these microwave resonators can be used as sensors for sorting dielectric objects. Techniques based upon this principle are in common use for measuring the dielectric and magnetic properties of materials at microwave frequencies for variety of applications. This thesis presents an approach of using Artificial Neural Networks to detect material change in a rectangular cavity. The method is based on the theory of the perturbation of cavity resonators where a change in the resonant frequencies of the cavity is directly proportional to the dielectric constant of the inserted objects. A rectangular cavity test fixture was built and excited with a monopole antenna. The cavity was filled with different materials, and the reflection coefficient of each material was measured over a wide range of frequencies. An intelligent systems approach using an artificial neural network (ANN) methodology was implemented for the automatic material change detection. To develop an automatic detection model, a multi-layer perceptron (MLP) was designed with one hidden layer and gradient descent back-propagation (BP) learning algorithm was used for the ANN training. The network training process was performed in an off-line mode, and after the training process was accomplished, the model was able to learn the rules without knowing any algorithm for automatic detection.

Artificial neural networks (ANNs) are essentially a 'black box' technology. The lack of an explanation component prevents the full and complete exploitation of this form of machine learning. During the mid 1990's the field of 'rule extraction' emerged. Rule extraction techniques attempt to derive a human comprehensible explanation structure from a trained ANN. Andrews et.al. (1995) proposed the following reasons for extending the ANN paradigm to include a rule extraction facility: * provision of a user explanation capability * extension of the ANN paradigm to 'safety critical' problem domains * software verification and debugging of ANN components in software systems * improving the generalization of ANN solutions * data exploration and induction of scientific theories * knowledge acquisition for symbolic AI systems An allied area of research is that of 'rule refinement'. In rule refinement an initial rule base, (i.e. what may be termed `prior knowledge') is inserted into an ANN by prestructuring some or all of the network architecture, weights, activation functions, learning rates, etc. The rule refinement process then proceeds in the same way as normal rule extraction viz (1) train the network on the available data set(s); and (2) extract the `refined' rules. Very few ANN techniques have the capability to act as a true rule refinement system. Existing techniques, such as KBANN, (Towell & Shavlik, (1993), are limited in that the rule base used to initialize the network must be a nearly complete, and the refinement process is limited to modifying antecedents. The limitations of existing techniques severely limit their applicability to real world problem domains. Ideally, a rule refinement technique should be able to deal with incomplete initial rule bases, modify antecedents, remove inaccurate rules, and add new knowledge by generating new rules. The motivation for this research project was to develop such a rule refinement system and to investigate its efficacy when applied to both nearly complete and incomplete problem domains. The premise behind rule refinement is that the refined rules better represent the actual domain theory than the initial domain theory used to initialize the network. The hypotheses tested in this research include: * that the utilization of prior domain knowledge will speed up network training, * produce smaller trained networks, * produce more accurate trained networks, and * bias the learning phase towards a solution that 'makes sense' in the problem domain. In 1998 Geva, Malmstrom, & Sitte, (1998) described the Local Cluster (LC) Neural Net. Geva et.al. (1998) showed that the LC network was able to learn / approximate complex functions to a high degree of accuracy. The hidden layer of the LC network is comprised of basis functions, (the local cluster units), that are composed of sigmoid based 'ridge' functions. In the General form of the LC network the ridge functions can be oriented in any direction. We describe RULEX, a technique designed to provide an explanation component for its underlying Local Cluster ANN through the extraction of symbolic rules from the weights of the local cluster units of the trained ANN. RULEX exploits a feature, ie, axis parallel ridge functions, of the Restricted Local Cluster (Geva , Andrews & Geva 2002), that allow hyper-rectangular rules of the form IF ∀ 1 ≤ i ≤ n : xi ∈ [ xi lower , xi upper ] THEN pattern belongs to the target class to be easily extracted from local functions that comprise the hidden layer of the LC network. RULEX is tested on 14 applications available in the public domain. RULEX results are compared with a leading machine learning technique, See5, with RULEX generally performing as well as See5 and in some cases outperforming See5 in predictive accuracy. We describe RULEIN, a rule refinement technique that allows symbolic rules to be converted into the parameters that define local cluster functions. RULEIN allows existing domain knowledge to be captured in the architecture of a LC ANN thus facilitating the first phase of the rule refinement paradigm. RULEIN is tested on a variety of artificial and real world problems. Experimental results indicate that RULEIN is able to satisfy the first requirement of a rule refinement technique by correctly translating a set of symbolic rules into a LC ANN that has the same predictive bahaviour as the set of rules from which it was constructed. Experimental results also show that in the cases where a strong domain theory exists, initializing an LC network using RULEIN generally speeds up network training, produces smaller, more accurate trained networks, with the trained network properly representing the underlying domain theory. In cases where a weak domain theory exists the same results are not always apparent. Experiments with the RULEIN / LC / RULEX rule refinement method show that the method is able to remove inaccurate rules from the initial knowledge base, modify rules in the initial knowledge base that are only partially correct, and learn new rules not present in the initial knowledge base. The combination of RULEIN / LC / RULEX thus is shown to be an effective rule refinement technique for use with a Restricted Local Cluster network.

The contributions of this thesis are an implicit modelling method for the coupled constitutive relations and an energy-based method for material modelling. The two developed methods utilise implicit models to represent material constitutive relations without the requirement of physical parameters. The first method is developed to model coupled constitutive relations using state-space representation with neural networks. State-space representation is employed to express the desired relations in a compact fashion while simultaneously providing the capability of modelling rate- and/or path-dependent behaviour. The employment of neural networks with the generalised state-space representation results in a single implicit model that can be adapted for a broad range of constitutive behaviours. The performance and applicability of the method are highlighted through the applications for various constitutive behaviour of piezoelectric materials, including the effects of hysteresis and cyclic degradation. An energy-based method is subsequently developed for implicit constitutive modelling by utilising the energy principle on a deformed continuum. Two formulations of the proposed method are developed for the modelling of materials with varying nature in directional properties. The first formulation is based on an implicit strain energy density function, represented by a neural network with strain invariants as input, to derive the desired stress-strain relations. The second formulation consists of the derivation of an energy-based performance function for training a neural network that represents the stress-strain relations. The requirement of deriving stress is eliminated in both formulations and this facilitates the use of advanced experimental setup, such as multi-axial load tests or non-standard specimens, to produce the most information for constitutive modelling from a single experiment. A series of numerical studies -- including validation problems and practical cases with actual experimental setup -- have been conducted, the results of which demonstrate the applicability and effectiveness of the proposed method for constitutive modelling on a continuum basis.

<p>Simulations are widely used for analysis and design of complex systems. Real-world complex systems are often too complex to be expressed with tractable mathematical formulations. Therefore simulations are often used instead of mathematical formulations because of their flexibility and ability to model real-world complex systems in some detail. Simulation models can often be complex and slow which lead to the development of simulation meta-models that are simpler and faster models of complex simulation models. Artificial neural networks (ANNs) have been studied for use as simulation meta-models with different results. This final year project further studies the use of ANNs as simulation meta-models by comparing the predictability of five different neural network architectures: feed-forward-, generalized feed-forward-, modular-, radial basis- and Elman artificial neural networks where the underlying simulation is of complex production system. The results where that all architectures gave acceptable results even though it can be said that Elman- and feed-forward ANNs performed the best of the tests conducted here. The difference in accuracy and generalization was considerably small.</p>

Στην παρούσα διπλωματική εργασία μελετάμε την ικανότητα των τεχνητών νευρωνικών δικτύων στη σύνθεση μουσικής και την αναγνώριση μουσικού συνθέτη. Συγκεκριμένα, στο πρώτο κεφάλαιο κάνουμε μία εισαγωγή στα τεχνητά νευρωνικά δίκτυα και ειδικά σε αυτά που χρησιμοποιούνται στα επόμενα κεφάλαια. Γίνεται αναφορά στα βασικά είδη των ΤΝΔ που υπάρχουν, εμπρόσθιας τροφοδότησης και αναδραστικά και περιγράφονται οι αλγόριθμοι εκπαίδευσής τους. Εξηγούμε την ικανότητα των αναδραστικών νευρωνικών δικτύων να έχουν δυναμική μνήμη, σε αντίθεση με αυτά που είναι εμπρόσθιας τροφοδότησης, πράγμα που τα καθιστά ικανά στην πρόβλεψη χρονοσειρών. Αυτή η ικανότητα των αναδραστικών δικτύων σε συνδυασμό με το γεγονός ότι ένα μουσικό κομμάτι μπορεί να χαρακτηριστεί σαν μία αλληλουχία γεγονότων χρονικής συνοχής (χρονοσειρά) δημιούργησε ένα ερευνητικό ρεύμα προς την κατεύθυνση της σύνθεσης μουσικής με τη χρήση ανδραστικών τεχνητών νευρωνικών δικτύων. Στο δεύτερο κεφάλαιο κάνουμε μία αναφορά στην αλγοριθμική σύνθεση μουσικής, ιδιαίτερα με χρήση πινάκων μετάβασης. Έπειτα ακολουθεί η περιγραφή του CONCERT, ενός αναδραστικού νευρωνικού δικτύου που κατασκευάστηκε για να συνθέτει μουσική με πρόβλεψη νότας προς νότα. Αναλύουμε επίσης την μοντελοποίηση των μουσικών αντικειμένων για την επεξεργασία και αναπαράστασή τους από το CONCERT η οποία βασίζεται σε ψυχοακουστικούς περιορισμούς αντίληψης των μουσικών αντικειμένων από τους ανθρώπους. Εξηγούμε τον τρόπο που εκπαιδεύεται το CONCERT έτσι ώστε να έχει όσο το δυνατόν μεγαλύτερη μνήμη και περιγράφουμε τις επιδόσεις του σε διάφορες δοκιμές που έγιναν, από την εκμάθηση μίας διατονικής κλίμακας μέχρι ενός κομματιού του J. S. Bach. Παρατηρώντας την ικανότητα του CONCERT να αντιλαμβάνεται τοπικές δομές (μοτίβα φράσεις) μα όχι καθολικές (μέρη του μουσικού κομματιού) αναφερόμαστε στην τεχνική της περιορισμένης περιγραφής που αποτελεί μια προσπάθεια για εκπαίδευση του δικτύου έτσι ώστε να αντιλαμβάνεται το μουσικό κομμάτι σε μία μεγαλύτερη κλίμακα. Στο τέλος του δευτέρου κεφαλαίου εξετάζουμε τη συνολική επίδοση του CONCERT και αναλύουμε τις κατευθύνσεις προς τις οποίες θα μπορούσαμε να κινηθούμε για τη βελτίωση των αποτελεσμάτων. Στο τρίτο κεφάλαιο αναφερόμαστε στην αναγνώριση του συνθέτη ενός μουσικού κομματιού με τη χρήση τεχνητών νευρωνικών δικτύων πάνω στην παρτιτούρα του κομματιού αυτού. Αρχικά γίνεται μία συζήτηση γύρω από το ποια στοιχεία της παρτιτούρας θεωρούμε σημαντικά, ποια από αυτά μπορούμε και ποια έχει νόημα να μοντελοποιήσουμε έτσι ώστε ένα ΤΝΔ να μπορεί να κάνει πρόβλεψη. Αναλύονται οι τεχνικές λεπτομέρειες των στοιχείων που χρειαζόμαστε για τη μοντελοποίηση μιας παρτιτούρας στον υπολογιστή και στη συνέχεια αναφερόμαστε στα δύο πειράματα που ελέγχουν την ορθότητα και αποτελεσματικότητα της παραπάνω προσέγγισης. Το ποια κομμάτια χρησιμοποιήθηκαν και από ποιους συνθέτες δε θα μπορούσε να είναι τυχαίο καθώς πρέπει να ικανοποιούνται διάφορες συνθήκες στατιστικής ομοιομορφίας έτσι ώστε η απάντηση του ΤΝΔ να είναι όσο το δυνατόν πιο αμερόληπτη. Αυτές οι συνθήκες, καθώς και οι κίνδυνοι που υπάρχουν σε πιθανή παράληψή τους εξηγούνται πριν τα πειράματα. Το πρώτο πείραμα πραγματεύεται την αναγνώριση συνθέτη ενός κομματιού που συντέθηκε από τον Chopin ή όχι (δηλαδή από τους Beethoven ή Mozart) ενώ στο δεύτερο οι εμπλεκόμενοι συνθέτες είναι οι Bach και Handel. Δοκιμάζονται διάφορες αρχιτεκτονικές ΤΝΔ και μετρούμε τη μέση και τη βέλτιστη επίδοσή τους. Τέλος συζητάμε τα αποτελέσματα των δύο πειραμάτων καθώς και τροποποιήσεις είτε του ΤΝΔ είτε της μοντελοποίησης που διαλέξαμε για την αναπαράσταση της παρτιτούρας στον υπολογιστή έτσι ώστε να έχουμε καλύτερα αποτελέσματα. / In this work we study the capability of artificial neural networks for composing music and musical composer recognition. To this end, in the first chapter the neural networks are introduced, especially the forms of those that are used later on. A reference is being made to the basic forms of neural networks, feedforward (FNN) and recursive (RNN), and their training algorithms. We explain the ability of the RNNs to have dynamic memory, in contrast to FNNs, which makes them suitable for predicting time series. This ability combined to the fact that a musical piece can be considered as a time series has urged researchers to explore music composition through RNNs. In the second chapter algorithmic music composition is being described, especially with the use of Markov chains. Then we describe CONCERT, a RNN constructed for composing music with note by note prediction. We also analyze the representation of musical objects which is based in how humans perceive them. CONCERT is trained with different musical patterns (from diatonic scales to Bach pieces) and its composing ability is being discussed. The fact that CONCERT lacks in capturing the global structure of a piece is not changed with the use of reduced description, which is thoroughly described. The second chapter concludes with thoughts on how a RNN could capture the global structure of a piece. The third chapter is devoted to composer recognition with the use of FNNs. Firstly we discuss which elements of a score are useful and which of them we can represent such that a FNN can identify a composer. The techniques that we use for the computer modeling of the problem and the manipulation of the pieces are thoroughly described. Two experiments are presented, in the first one the FNN is called to recognize Chopin from Mozart and Beethoven and in the second Bach from Handel. Finally a discussion is made on the results of the above experiments and how we could optimize them.

The human olfactory system can classify new odors in a dynamic environment with varying odor complexity and concentration, while simultaneously reducing the influence of stable background odors. Replication of this capability has remained an active area of research over the past 3 decades and has great potential to advance medical diagnostics, environmental monitoring and industrial monitoring, among others. New methods for rapid dynamic temporal evaluation of chemical sensor arrays for the monitoring of analytes is explored in this work. One such method is high and low bandpass filtering of changing sensor responses; this is applied to reduce the effects of background noise and sensor drift over time. Processed sensor array responses, coupled with principal component analysis (PCA), will be used to develop a novel approach to classify odors in the presence of changing sensor responses associated with evolving odor concentrations. These methods will enable the removal of noise and drift, as well as facilitating the normalization to decouple classification patterns from intensity; lastly, PCA and artificial neural networks (ANNs) will be used to demonstrate the capability of this approach to function under dynamic conditions, where concentration is changing temporally. / February 2016

The operation of large industrial scale combustion systems, such as furnaces and boilers is increasingly dictated by emission legislation and requirements for improved efficiency. However, it can be exceedingly difficult and time consuming to gather the information required to improve original designs. Mathematical modelling techniques have led to the development of sophisticated furnace representations that are capable of representing combustion parameters. Whilst such data is ideal for design purposes, the current power of computing systems tends to generate simulation times that are too great to embed the models into online control strategies. The work presented in this thesis offers the possibility of replacing such mathematical models with suitably trained Artificial Neural Networks (ANNs) since they can compute the same outputs at a fraction of the model's speed, suggesting they could provide an ideal alternative in online control strategies. Furthermore, artificial neural networks have the ability to approximate and extrapolate making them extremely robust when encountering conditions not met previously. In addition to improving operational procedures, another approach to increasing furnace system efficiency is to minimise the waste heat energy produced during the combustion process. One very successful method involves the implementation of a heat exchanger system in the exiting gas flue stream, since this is predominantly the main source of heat loss. It can be exceptionally difficult to determine which heat exchanger is best suited for a particular application and it can prove an even more arduous task to control it effectively. Furthermore, there are many factors that alter the performance characteristics of a heat exchanger throughout the duration of its operational life, such as fouling or unexpected systematic faults. This thesis investigates the modelling of an experimental heat exchanger system via artificial neural networks with a view to aiding the design and selection process. Moreover, the work presented offers a means to control heat exchangers subject to varying operating conditions more effectively, thus promoting savings in both waste energy and time.

Master of Science / Department of Computer Science / William H. Hsu / Cryptocurrencies are digital currencies that have garnered significant investor attention in the financial markets. The aim of this project is to predict the daily price, particularly the daily high and closing price, of the cryptocurrency Bitcoin. This plays a vital role in making trading decisions. There exist various factors which affect the price of Bitcoin, thereby making price prediction a complex and technically challenging task. To perform prediction, we trained temporal neural networks such as time-delay neural networks (TDNN) and recurrent neural networks (RNN) on historical time series – that is, past prices of Bitcoin over several years. Features such as the opening price, highest price, lowest price, closing price, and volume of a currency over several preceding quarters were taken into consideration so as to predict the highest and closing price of the next day. We designed and implemented TDNNs and RNNs using the NeuroSolutions artificial neural network (ANN) development environment to build predictive models and evaluated them by computing various measures such as the MSE (mean square error), NMSE (normalized mean square error), and r (Pearson’s correlation coefficient) on a continuation of the training data from each time series, held out for validation.