Biophysically detailed representations of neural network models provide substantial insight to underlying neural processing mechanisms in the auditory systems of the brain. For simple biological systems the behavior can be represented by simple equations or flow charts. But for complex systems, more detailed descriptions of individual neurons and their synaptic connectivity are typically required. Creating extensive network models allows us to test hypotheses, apply specific manipulations that cannot be done experimentally and provide supporting evidence for experimental results. Several studies have been made on establishing realistic models of the cochlear nucleus (Manis and Campagnola, 2018; Eager et al., 2004), the part of the brainstem where sound signals enter the brain, both on individual neuron and networked structure levels. These models are based on both in vitro and in vivo physiological data, and they successfully demonstrate certain aspects of the neural processing of sound signals. Even though these models have been tested with tone bursts and isolated phonemes, the representation of speech in the cochlear nucleus and how it may support robust speech intelligibility remains to be explored with these detailed biophysical models. In this study, the basis of creating a biophysically detailed model of microcircuits in the cochlear nucleus is formed following the approach of Manis and Campagnola (2018). The focus of this thesis is more on bushy cell microcircuits. We have updated Manis and Campagnola (2018) model to take inputs from the new phenomenological auditory periphery model of Bruce et al. (2018). Different cell types in the cochlear nucleus are modelled by detailed cell models of Rothman and Manis (2003c) and updated Manis and Campagnola (2018) cell models. Networked structures are built out of them according to published anatomical and physiological data. The outputs of these networked structures are used to create post-stimulus-time-histograms (PSTH) and response maps to investigate the representation of tone bursts and average localized synchronized rate (ALSR) of phoneme 'e' and are compared to published physiological data (Blackburn and Sachs, 1990). / Thesis / Master of Applied Science (MASc)

We specify a class of non-linear and non-Gaussian models for which we estimate and forecast the conditional distributions with daily frequency. We use these forecasts to calculate VaR measures for three different equity markets (US, GB and Japan). These forecasts are evaluated on the basis of different statistical performance measures as well as on the basis of their economic costs that go along with the forecasted capital requirements. The results indicate that different performance measures generate different rankings of the models even within one financial market. We also find that for the three markets the improvement in the forecast by non-linear models over linear ones is negligible, while non-gaussian models significantly dominate the gaussian models. / Series: Report Series SFB "Adaptive Information Systems and Modelling in Economics and Management Science"

Robot control systems evolved with genetic algorithms traditionally take the form of floating-point neural network models. This thesis proposes that digital control systems, such as quantised neural networks and logical networks, may also be used for the task of robot control. The inspiration for this is the observation that the dynamics of discrete networks may contain cyclic attractors which generate rhythmic behaviour, and that rhythmic behaviour underlies the central pattern generators which drive lowlevel motor activity in the biological world. To investigate this a series of experiments were carried out in a simulated physically realistic 3D world. The performance of evolved controllers was evaluated on two well known control tasks—pole balancing, and locomotion of evolved morphologies. The performance of evolved digital controllers was compared to evolved floating-point neural networks. The results show that the digital implementations are competitive with floating-point designs on both of the benchmark problems. In addition, the first reported evolution from scratch of a biped walker is presented, demonstrating that when all parameters are left open to evolutionary optimisation complex behaviour can result from simple components.

Classifier ensembling research has been one of the most active areas of machine learning for a long period of time. The main aim of generating combined classifier ensembles is to improve the prediction accuracy in comparison to using an individual classifier. A combined classifiers ensemble can improve the prediction results by compensating for the individual classifier weaknesses in certain areas and benefiting from better accuracy of the other ensembles in the same area. In this thesis, different algorithms are proposed for designing classifier ensemble combiners. The existing methods such as averaging, voting, weighted average, and optimised weighted method does not increase the accuracy of the combiner in comparison to the proposed advanced methods such as genetic programming and the coalition method. The different methods are studied in detail and analysed using different databases. The aim is to increase the accuracy of the combiner in comparison to the standard stand-alone classifiers. The proposed methods are based on generating a combiner formula using genetic programming, while the coalition is based on estimating the diversity of the classifiers such that a coalition is generated with better prediction accuracy. Standard accuracy measures are used, namely accuracy, sensitivity, specificity and area under the curve, in addition to training error accuracies such as the mean square error. The combiner methods are compared empirically with several stand-alone classifiers using neural network algorithms. Different types of neural network topologies are used to generate different models. Experimental results show that the combiner algorithms are superior in creating the most diverse and accurate classifier ensembles. Ensembles of the same models are generated to boost the accuracy of a single classifier type. An ensemble of 10 models of different initial weights is used to improve the accuracy. Experiments show a significant improvement over a single model classifier. Finally, two combining methods are studied, namely the genetic programming and coalition combination methods. The genetic programming algorithm is used to generate a formula for the classifiers’ combinations, while the coalition method is based on a simple algorithm that assigns linear combination weights based on the consensus theory. Experimental results of the same databases demonstrate the effectiveness of the proposed methods compared to conventional combining methods. The results show that the coalition method is better than genetic programming.

A useful compiler has been designed that takes a high level neural network specification and constructs a low level configuration file explicitly specifying all network parameters and connections. The neural network model for which this compiler was designed is the adaptrode neuronal model, and the configuration file created can be used by the Adnet simulation engine to perform network experiments. The specification language is very flexible and provides a general framework from which almost any network wiring configuration may be created. While the compiler was created for the specialized adaptrode model, the wiring specification algorithms could also be used to specify the connections in other types of networks.

Written skills are an essential evaluation criterion for a student’s creativity, knowledge, and intellect. Consequently, academic writing is a common part of university and college admissions applications, standardized tests, and classroom assessments. However, the task for teachers is quite daunting when it comes to essay scoring. Then Automated Essay Scoring may be a helpful tool in the decision-making by the teacher.  There have been many successful models with supervised or unsupervised machine learning algorithms in the eld of Automated Essay Scoring. This thesis work makes a comparative study among various neural network models with supervised machine learning algorithms and different linguistic feature combinations. It also proves that the same linguistic features are applicable to more than one language.  The models studied in this experiment include TextCNN, TextRNN_LSTM, Tex- tRNN_GRU, and TextRCNN trained with the essays from the Automated Student Assessment Prize (ASAP) from Kaggle competitions. Each essay is represented with linguistic features measuring linguistic complexity. Those features are divided into four groups: count-based, morphological, syntactic, and lexical features, and the four groups of features can form a total of 14 combinations.  The models are evaluated via three measurements: Accuracy, F1 score, and Quadratic Weighted Kappa. The experimental results show that models trained only with count-based features outperform the models trained using other feature combinations. In addition, TextRNN_LSTM performs best, with an accuracy of 54.79%, an F1 score of 0.55, and a Quadratic Weighted Kappa of 0.59, which beats the statistically-based baseline models.

No description available.

Neural network models such as Transformer-based BERT, mBERT and RoBERTa are achieving impressive performance (Devlin et al., 2019; Lewis et al., 2020; Liu et al., 2019; Raffel et al., 2020; Y. Sun et al., 2019), but we still know little about their inner working due to the complex technique like multi-head self-attention they implement. Attention is commonly taken as a crucial way to explain the model outputs, but there are studies argue that attention may not provide faithful and reliable explanations in recent years (Jain and Wallace, 2019; Pruthi et al., 2020; Serrano and Smith, 2019; Wiegreffe and Pinter, 2019). Bastings and Filippova (2020) then propose that saliency may give better model interpretations since it is designed to find which token contributes to the prediction, i.e. the exact goal of explanation.  In this thesis, we investigate the extent to which syntactic structure is reflected in BERT, mBERT and RoBERTa trained on English and Chinese by using a gradient-based saliency method introduced by Simonyan et al. (2014). We examine the dependencies that our models and baselines predict.  We find that our models can predict some dependencies, especially those that have shorter mean distance and more fixed position of heads and dependents, even though all our models can handle global dependencies in theory. Besides, BERT usually has higher overall accuracy on connecting dependents to their corresponding heads, followed by mBERT and RoBERTa. Yet all the three model in fact have similar results on individual relations. Moreover, models trained on English have better performances than models trained on Chinese, possibly because of the flexibility of Chinese language.

The data-driven models used for the design of powertrain controllers are typically based on the data obtained from steady-state experiments. However, they are only valid under stable conditions and do not provide any information on the dynamic behaviour of the system. In order to capture this behaviour, dynamic modelling techniques are intensively studied to generate alternative solutions for engine mapping and calibration problem, aiming to address the need to increase productivity (reduce development time) and to develop better models for the actual behaviour of the engine under real-world conditions. In this thesis, a dynamic modelling approach is presented undertaken for the prediction of NOx emissions for a 2.0 litre Diesel engine, based on a coupled pre-validated virtual Diesel engine model (GT- Suite ® 1-D air path model) and in-cylinder combustion model (CMCL ® Stochastic Reactor Model Engine Suite). In the context of the considered Engine Simulation Framework, GT Suite + Stochastic Reactor Model (SRM), one fundamental problem is to establish a real time stochastic simulation capability. This problem can be addressed by replacing the slow combustion chemistry solver (SRM) with an appropriate NOx surrogate model. The approach taken in this research for the development of this surrogate model was based on a combination of design of dynamic experiments run on the virtual diesel engine model (GT- Suite), with a dynamic model fitted for the parameters required as input to the SRM, with a zonal design of experiments (DoEs), using Optimal Latin Hypercubes (OLH), run on the SRM model. A response surface model was fitted on the predicted NOx from the SRM OLH DoE data. This surrogate NOx model was then used to replace the computationally expensive SRM simulation, enabling real-time simulations of transient drive cycles to be executed. The performance of the approach was validated on a simulated NEDC drive cycle, against experimental data collected for the engine case study. The capability of methodology to capture the transient trends of the system shows promising results and will be used for the development of global surrogate prediction models for engine-out emissions.

This study presents the implementation of Middle East Technical University Neural Network (METU-NN) models for the ionospheric forecasting together with worldwide usage capability of the Internet. Furthermore, an attempt is made to include expert information in the Neural Network (NN) model in the form of neurofuzzy network (NFN). Middle East Technical University Neurofuzzy Network (METU-NFN) modeling approach is developed which is the first attempt of using a neurofuzzy model in the ionospheric forecasting studies. The Web based applications developed in this study have the ability to be customized such that other NN and NFN models including METU-NFN can also be adapted. The NFN models developed in this study are compared with the previously developed and matured METU-NN models. At this very early stage of employing neurofuzzy models in this field, ambitious objectives are not aimed. Applicability of the neurofuzzy systems on the ionospheric forecasting studies is only demonstrated. Training and operating METU-NN and METU-NFN models under equal conditions and with the same data sets, the cross correlation of obtained and measured values are 0.9870 and 0.9086 and the root mean square error (RMSE) values of 1.7425 TECU and 4.7987 TECU are found by operating METU-NN and METU-NFN models respectively. The results obtained by METU-NFN model is close to those found by METU-NN model. These results are reasonable enough to encourage further studies on neurofuzzy models to benefit from expert information. Availability of these models which already attracted intense international attention will greatly help the related scientific circles to use the models. The models can be architecturally constructed, trained and operated on-line. To the best of our knowledge this is the first application that gives the ability of on-line model usage with these features. Applicability of NFN models to the ionospheric forecasting is demonstrated. Having ample flexibility the constructed model enables further developments and improvements. Other neurofuzzy systems in the literature might also lead to better achievements.