Neural networks play an important role in the execution of goal-oriented paradigms. They offer flexibility, adaptability and versatility, so that a variety of approaches may be used to meet a specific goal, depending upon the circumstances and the requirements of the design specifications. Development of higher-order neural units with higher-order synaptic operations will open a new window for some complex problems such as control of aerospace vehicles, pattern recognition, and image processing. The neural models described in this thesis consider the behavior of a single neuron as the basic computing unit in neural information processing operations. Each computing unit in the network is based on the concept of an idealized neuron in the central nervous system (CNS). Most recent mathematical models and their architectures for neuro-control systems have generated many theoretical and industrial interests. Recent advances in static and dynamic neural networks have created a profound impact in the field of neuro-control. Neural networks consisting of several layers of neurons, with linear synaptic operation, have been extensively used in different applications such as pattern recognition, system identification and control of complex systems such as flexible structures, and intelligent robotic systems. The conventional linear neural models are highly simplified models of the biological neuron. Using this model, many neural morphologies, usually referred to as multilayer feedforward neural networks (MFNNs), have been reported in the literature. The performance of the neurons is greatly affected when a layer of neurons are implemented for system identification, pattern recognition and control problems. Through simulation studies of the XOR logic it was concluded that the neurons with linear synaptic operation are limited to only linearly separable forms of pattern distribution. However, they perform a variety of complex mathematical operations when they are implemented in the form of a network structure. These networks suffer from various limitations such as computational efficiency and learning capabilities and moreover, these models ignore many salient features of the biological neurons such as time delays, cross and self correlations, and feedback paths which are otherwise very important in the neural activity. In this thesis an effort is made to develop new mathematical models of neurons that belong to the class of higher-order neural units (HONUs) with higher-order synaptic operations such as quadratic and cubic synaptic operations. The advantage of using this type of neural unit is associated with performance of the neurons but the performance comes at the cost of exponential increase in parameters that hinders the speed of the training process. In this context, a novel method of representation of weight parameters without sacrificing the neural performance has been introduced. A generalised representation of the higher-order synaptic operation for these neural structures was proposed. It was shown that many existing neural structures can be derived from this generalized representation of the higher-order synaptic operation. In the late 1960s, McCulloch and Pitts modeled the stimulation-response of the primitive neuron using the threshold logic. Since then, it has become a practice to implement the logic circuits using neural structures. In this research, realization of the logic circuits such as OR, AND, and XOR were implemented using the proposed neural structures. These neural structures were also implemented as neuro-controllers for the control problems such as satellite attitude control and model reference adaptive control. A comparative study of the performance of these neural structures compared to that of the conventional linear controllers has been presented. The simulation results obtained in this research were applicable only for the simplified model presented in the simulation studies.

Nature has created complex biological structures that exhibit intelligent behaviour through an evolutionary process. Thus, intelligence and evolution are intimately connected. This has inspired evolutionary computation (EC) that simulates the evolutionary process to develop powerful techniques such as genetic algorithms (GAs), genetic programming (GP), evolutionary strategies (ES) and evolutionary programming (EP) to solve real-world problems in learning, control, optimization and classification. GP discovers the relationship among data and expresses it as a LISP-S expression i.e., a computer program. Thus the goal of program discovery as a solution for a problem is addressed by GP in the framework of evolutionary computation. In this thesis, we address for the first time the problem of applying GP to mu1ticategory pattern classification. In supervised pattern classification, an input vector of m dimensions is mapped onto one of the n classes. It has a number of application areas such as remote sensing, medical diagnosis etc., A supervised classifier is developed by using a training set that contains representative samples of various classes present in the application. Supervised classification has been done earlier with maximum likelihood classifier: neural networks and fuzzy logic. The major considerations in applying GP to pattern classification are listed below: (i) GP-based techniques are data distribution-free i.e., no a priori knowledge is needed abut the statistical distribution of the data or no assumption such as normal distribution for data needs to be made as in MLC. (ii) GP can directly operate on the data in its original form. (iii) GP can detect the underlying but unknown relationship that mists among data and express it as a mathematical LISP S-expression. The generated LISP S-expressions can be directly used in the application environment. (iv) GP can either discover the most important discriminating features of a class during evolution or it requires minor post-processing of the LISP-S expression to discover the discriminant features. In a neural network, the knowledge learned by the neural network about the data distributions is embedded in the interconnection weights and it requires considerable amount of post-processing of the weights to understand the decision of the neural network. In 2-category pattern classification, a single GP expression is evolved as a discriminant function. The output of the GP expression can be +l for samples of one class and -1 for samples of the other class. When the GP paradigm is applied to an n-class problem, the following questions arise: Ql. As a typical GP expression returns a value (+l or -1) for a 2-class problem, how does one apply GP for the n-class pattern classification problem? Q2. What should be the fitness function during evolution of the GP expressions? Q3. How does the choice of a function set affect the performance of GP-based classification? Q4. How should training sets be created for evaluating fitness during the evolution of GP classifier expressions? Q5. How does one improve learning of the underlying data distributions in a GP framework? Q6. How should conflict resolution be handled before assigning a class to the input feature vector? Q7. How does GP compare with other classifiers for an n-class pattern classification problem? The research described here seeks to answer these questions. We show that GP can be applied to an n-category pattern classification problem by considering it as n 2-class problems. The suitability of this approach is demonstrated by considering a real-world problem based on remotely sensed satellite images and Fisher's Iris data set. In a 2-class problem, simple thresholding is sufficient for a discriminant function to divide the feature space into two regions. This means that one genetic programming classifier expression (GPCE) is sufficient to say whether or not the given input feature vector belongs to that class; i.e., the GP expression returns a value (+1 or -1). As the n-class problem is formulated as n 2-class problems, n GPCEs are evolved. Hence, n GPCE specific training sets are needed to evolve these n GPCEs. For the sake of illustration, consider a 5-class pat tern classification problem. Let n, be the number of samples that belong to class j, and N, be the number of samples that do not belong to class j, (j = 1,..., 5). Thus, N1=n2+n3+n4+n5 N2=n1+n3+n4+n5 N3=n1+n2+n4+n5 N4=n1+n2+n3+n5 N5=n1+n2+n3+n4 Thus, When the five class problem is formulated as five 2-class problems. we need five GPCEs as discriminant functions to resolve between n1 and N1, n2 and N2, n3 and N3, n4 and N4 and lastly n5 and N5. Each of these five 2-class problems is handled as a separate 2-class problem with simple thresholding. Thus, GPCE# l resolves between samples of class# l and the remaining n - 1 classes. A training set is needed to evaluate the fitness of GPCE during its evolution. If we directly create the training set, it leads to skewness (as n1 < N1). To overcome the skewness, an interleaved data format is proposed for the training set of a GPCE. For example, in the training set of GPCE# l, samples of class# l are placed alternately between samples of the remaining n - 1 classes. Thus, the interleaved data format is an artifact to create a balanced training set. Conventionally, all the samples of a training set are fed to evaluate the fitness of every member of the population in each generation. We call this "global" learning 3s GP tries to learn the entire training set at every stage of the evolution. We have introduced incremental learning to simplify the task of learning for the GP paradigm. A subset of the training set is fed and the size of the subset is gradually increased over time to cover the entire training data. The basic motivation for incremental learning is to improve learning during evolution as it is easier to learn a smaller task and then to progress from a smaller task to a bigger task. Experimental results are presented to show that the interleaved data format and incremental learning improve the performance of the GP classifier. We also show that the GPCEs evolved with an arithmetic function set are able to track variation in the input better than GPCEs evolved with function sets containing logical and nonlinear elements. Hence, we have used arithmetic function set, incremental learning, and interleaved data format to evolve GPCEs in our simulations. AS each GPCE is trained to recognize samples belonging to its own class and reject samples belonging to other classes a strength of association measure is associated with each GPCE to indicate the degree to which it can recognize samples belonging to its own class. The strength of association measures are used for assigning a class to an input feature vector. To reduce misclassification of samples, we also show how heuristic rules can be generated in the GP framework unlike in either MLC or the neural network classifier. We have also studied the scalability and generalizing ability of the GP classifier by varying the number of classes. We also analyse the performance of the GP classifier by considering the well-known Iris data set. We compare the performance of classification rules generated from the GP classifier with those generated from neural network classifier, (24.5 method and fuzzy classifier for the Iris data set. We show that the performance of GP is comparable to other classifiers for the Iris data set. We notice that the classification rules can be generated with very little post-processing and they are very similar to the rules generated from the neural network and C4.5 for the Iris data set. Incremental learning influences the number of generations available for GP to learn the data distribution of classes whose d is -1 in the interleaved data format. This is because the samples belonging to the true class (desired output d is +1) are alternately placed between samples belonging to other classes i.e., they are repeated to balance the training set in the interleaved data format. For example, in the evolution of GPCE for class# l, the fitness function can be fed initially with samples of class#:! and subsequently with the samples of class#3, class#4 and class#. So in the evaluation of the fitness function, the samples of class#kt5 will not be present when the samples of class#2 are present in the initial stages. However, in the later stages of evolution, when samples of class#5 are fed, the fitness function will utilize the samples of both class#2 and class#5. As learning in evolutionary computation is guided by the evaluation of the fitness function, GPCE# l gets lesser number of generations to learn how to reject data of class#5 as compared to the data of class#2. This is because the termination criterion (i.e., the maximum number of generations) is defined a priori. It is clear that there are (n-l)! Ways of ordering the samples of classes whose d is -1 in the interleaved data format. Hence a heuristic is presented to determine a possible order to feed data of different classes for the GPCEs evolved with incremental learning and interleaved data format. The heuristic computes an overlap index for each class based on its spatial spread and distribution of data in the region of overlap with respect to other classes in each feature. The heuristic determines the order in which classes whose desired output d is –1 should be placed in each GPCE-specific training set for the interleaved data format. This ensures that GP gets more number of generations to learn about the data distribution of a class with higher overlap index than a class with lower overlap index. The ability of the GP classifier to learn the data distributions depends upon the number of classes and the spatial spread of data. As the number of classes increases, the GP classifier finds it difficult to resolve between classes. So there is a need to partition the feature space and identify subspaces with reduced number of classes. The basic objective is to divide the feature space into subspaces and hence the data set that contains representative samples of n classes into subdata sets corresponding to the subspaces of the feature space, so that some of the subdata sets/spaces can have data belonging to only p classes (p < n). The GP classifier is then evolved independently for the subdata sets/spaces of the feature space. This results in localized learning as the GP classifier has to learn the data distribution in only a subspace of the feature space rather than in the entire feature space. By integrating the GP classifier with feature space partitioning (FSP), we improve classification accuracy due to localized learning. Although serial computers have increased steadily in their performance, the quest for parallel implementation of a given task has continued to be of interest in any computationally intensive task since parallel implementation leads to a faster execution than a serial implementation As fitness evaluation, selection strategy and population structures are used to evolve a solution in GP, there is scope for a parallel implementation of GP classifier. We have studied distributed GP and massively parallel GP for our approach to GP-based multicategory pattern classification. We present experimental results for distributed GP with Message Passing Interface on IBM SP2 to highlight the speedup that can be achieved over the serial implementation of GP. We also show how data parallelism can be used to further speed up fitness evaluation and hence the execution of the GP paradigm for multicategory pat tern classification. We conclude that GP can be applied to n-category pattern classification and its potential lies in its simplicity and scope for parallel implementation. The GP classifier developed in this thesis can be looked upon as an addition to the earlier statistical, neural and fuzzy approaches to multicategory pattern classification.

The growth in the use of machine learning in different areas has revealed challenging classification problems that require robust systems. Multiple Classier Systems (MCSs) have attracted interest from researchers as a method that could address such problems. Optimizing the fitness of an MCS improves its, robustness. The lack of an analysis for MCSs from a fitness perspective is identified. To fill this gap, an analytic model from this perspective is derived mathematically by extending the error analysis introduced by Brown and Kuncheva in 2010. The model relates the fitness of an MCS to the average accuracy, positive-diversity, and negative-diversity of the classifiers that constitute the MCS. The model is verified using a statistical analysis of a Monte-Carlo based simulation. This shows the significance of the indicated relationships by the model. This model provides guidelines for developing robust MCSs. It enables the selection of classifiers which compose an MCS with an improved fitness while improving computational cost by avoiding local calculations. The usefulness of the model for designing classification systems is investigated. A new measure consisting of the accuracy and positive-diversity is developed. This measure evaluates fitness while avoiding many calculations compared to the regular measures. A new system (Gadapt) is developed. Gadapt combines machine learning and genetic algorithms to define subsets of the feature space that closely match true class regions. It uses the new measure as a multi-objective criterion for a multi-objective genetic algorithm to identify the MCSs those create the subsets. The design of Gadapt is validated experimentally. The usefulness of the measure and the method of determining the subsets for the performance of Gadapt are examined based on five generated data sets that represent a wide range of problems. The robustness of Gadapt to small amounts of training data is evaluated in comparison with five existing systems on four benchmark data sets. The performance of Gadapt is evaluated in comparison with eleven existing systems on nine benchmark data sets. The analysis of the experiment results supports the validity of the Gadapt design and the outperforming of Gadapt on the existing systems in terms of robustness and performance.

The aim of this thesis is to develop a stereo vision system to locate and classify objects moving on a conveyor belt. The vision system determines the locations of the objects with respect to a world coordinate system and class of the objects. In order to estimate the locations of the objects, two cameras placed at different locations are used. Image processing algorithms are employed to extract some features of the objects. These features are fed to stereo matching and classifier algorithms. The results of stereo matching algorithm are combined with the calibration parameters of the cameras to determine the object locations. Pattern classification techniques (Bayes and Nearest Neighbor classifiers) are used to classify the objects. The linear velocity of the objects is determined by using an encoder mounted to the shaft of the motor driving the conveyor belt. A robot can plan a sequence of motion to pick the object from the conveyor belt by using the output of the proposed system.

Three-dimensional movement analysis was used to evaluate head and neck movement in patients with neck pain and matched controls. The aims were to further develop biomechanical models of head and neck kinematics, to investigate differences between subjects with non-specific neck pain and whiplash associated disorders (WAD), and to evaluate the potential of objective movement analysis as a decision support during diagnosis and follow-up of patients with neck pain. Fast, repetitive head movements (flexion, extension, rotation to the side) were studied in a group of 59 subjects with WAD and 56 controls. A back propagation artificial neural network classified vectors of collected movement variables from each individual according to group membership with a predictivity of 89%. The helical axis for head movement were analyzed in two groups of neck pain patients (21 with non-specific neck pain and 22 with WAD) and 24 matched controls. A moving time window with a cut-off angle of 4° was used to calculate finite helical axes. The centre of rotation of the finite axes (CR) was derived as the 3D intersection point of the finite axes. A downward migration of the axis during flexion/extension and a change of axis direction towards the end of the movements were observed. CR was at its most superior position during side rotations and at its most inferior during ball catching. This could relate to that side rotation was mainly done in the upper spine, while all cervical vertebrae were recruited to stabilize the head in the more complex catching task. Changes in movement strategy were observed in the neck pain groups: Neck pain subjects had lower mean velocities and ranges of movements as compared with controls during ball catching, which could relate to a stiffer body position in neck pain patients in order to stabilize the neck. In addition, the WAD group had a displaced axis position during head repositioning after flexion, while CR was displaced during fast side rotations in the non-specific neck pain group. Pain intensity correlated with axis and CR position, and may be one reason for the movement strategy changes. Increased amount of irregularities in the trajectory of the axis was found in the WAD group during head repositioning, fast repetitive head movements and catching. This together with an increased constant repositioning error during repositioning after flexion indicated motor control disturbances. A higher group standard deviation in neck pain groups indicated heterogeneity among subjects in this disturbance. Wireless motion sensors and electro-oculography was used simultaneously, as an initial step towards a portable system and towards a method to quantify head-eye co-ordination deficits in individuals with WAD. Twenty asymptomatic control subjects and six WAD subjects with eye disturbances (e.g. dizziness and double vision) were studied. The trial-to-trial repeatability was moderate to high for all evaluated variables (single intraclass correlation coefficients &gt;0.4 in 28 of 32 variables). The WAD subjects demonstrated decreased head velocity, decreased range of head movement during gaze fixation and lowered head stability during head-eye co-ordination as possible deficits. In conclusion, kinematical analyses have a potential to be used as a support for physicians and physiotherapists for diagnosis and follow-up of neck pain patients. Specifically, the helical axis method gives information about how the movement is performed. However, a flexible motion capture system (for example based on wireless motion sensors) is needed. Combined analysis of several variables is preferable, as patients with different neck pain disorders seem to be a heterogeneous group.

The interest in the pattern classification field has increased due to challenging applications and also due to computational demands, specially when big datasets have to be analyzed. Statistical classification methods, as the based Bayes rules decision theory, apply the parameter estimation from a training dataset for recognizing different classes inside the dataset. In this work it is investigated the contribution of using the discrete wavelet transformation (DWT) for feature extraction during the classification process. From the scale coefficients of different decomposition levels, new training datasets, which are used in Bayesian classifier, are formed. For the one and two dimensional transforms the Daubechies wavelet family is considered. Three specifically wavelet functions are analyzed (Haar, Daubechies Db2 and Db8). Also, a hybrid methodology is proposed, in which 2D and 1D wavelet transformations are applied in consecutive stages of data analysis. For the evaluation of the one dimensional transform methodology, two different unidimensional datasets are used: one is composed by synthetic data, and the other is composed by network traffic data (DARPA1999 dataset). For the evaluation of 2D and hybrid methodologies two-dimensional data are considered. The two-dimensional data are images with different digital pictures with and without using ash light. One advantage of applying the hybrid methodology is the maintenance of the classification regularity and the increase of correct classification in some cases. / O interesse na área de classificação de padrões tem aumentado ultimamente devido a grande demanda computacional para a manipulação de grandes conjuntos de dados, e também ao aumento de aplicações desafidoras. Métodos de classificação estatística, como os métodos baseados na teoria de decisão das regras de Bayes, aplicam a abordagem de estimação de parâmetros a partir de um conjunto de dados de treinamento que definem as diferentes classes dentro de uma base de dados. Neste trabalho é avaliada a contribuição da transformada wavelet discreta (TWD) na extração de variáveis para a realização de classificação. A partir dos coeficientes de escala de diferentes níveis de TWD serão compostos novos conjuntos de dados de treinamento para serem aplicados em métodos de classificação Bayesiano. Para as transformadas uni- e bidimensionais são consideradas funções da família de wavelets ortonormais de Daubechies (Haar, Daubechies Db2 e Db8). Também é proposta uma metodologia híbrida para o tratamento de dados bidimensionais que compreende em aplicar tanto a transformada uni- quanto a bidimensional em estágios consecutivos da análise dos dados. Para a avaliação da metodologia de classificação associada à transformada unidimensional são utilizados dois conjuntos de dados unidimensionais diferentes: o primeiro é composto por dados gerados sinteticamente, e o outro é composto por informações de tráfego de dados em rede (base DARPA 1999). Para avaliar as metodologias bidimensional e híbrida são utilizados dados bidimensionais (imagens originadas de fotografias digitais, com e sem uso de ash). Uma vantagem da aplicação da metodologia híbrida é a manutenção da regularidade de classificação e o aumento nas classificações corretas em alguns casos.

Based on the targets outlined by the EU Horizon 2020 (H2020) framework, it is expected that heterogeneous networking will play a crucial role in delivering seamless end-to-end ubiquitous Internet access for users. In due course, the current GSM-Railway (GSM-R) will be deemed unsustainable, as the demand for packet-oriented services continues to increase. Therefore, the opportunity to identify a plausible replacement system conducted in this research study is timely and appropriate. In this research study, a hybrid satellite and terrestrial network for enabling ubiquitous Internet access in a high-speed train environment is investigated. The study focuses on the mobility management aspect of the system, primarily related to the handover management. A proposed handover strategy, employing the RACE II MONET and ITU-T Q.65 design methodology, will be addressed. This includes identifying the functional model (FM) which is then mapped to the functional architecture (FUA), based on the Q.1711 IMT-2000 FM. In addition, the signalling protocols, information flows and message format based on the adopted design methodology will also be specified. The approach is then simulated in OPNET and the findings are then presented and discussed. The opportunity of exploring the prospect of employing neural networks (NN) for handover is also undertaken. This study focuses specifically on the use of pattern classification neural networks to aid in the handover process, which is then simulated in MATLAB. The simulation outcomes demonstrated the effectiveness and appropriateness of the NN algorithm and the competence of the algorithm in facilitating the handover process.

Made available in DSpace on 2015-04-11T14:02:53Z (GMT). No. of bitstreams: 1 Mauricio Tia Ni Gong Lin.pdf: 636193 bytes, checksum: f576d0b7751120d8c4eaafb2517b1e22 (MD5) Previous issue date: 2006-02-10 / The growth of Linux operating system has taken it to become a worthy competitor to commercial software such as Microsoft s Windows and Sun s Solaris. Although the development and the improvement of several Linux s features, the problem related to Linux out of memory and the current mechanism used to solve it, named as OOM Killer, has brought a long discussion at Linux kernel community. The lack of scientific works related to OOM Killer process selection algorithm motivates this dissertation to propose a mechanism for identifying and classifying memory consumption patterns of Linux applications. Such mechanism is based on a neural network technique known as Self Organizing Maps. The development of a tool based on Self Organizing Maps presented the possibility of applying such approach for memory consumption patterns classification related to Linux applications use cases. / A evolução do sistema operacional Linux possibilitou que o mesmo se tornasse o principal concorrente dos sistemas operacionais do mercado como o Windows da Microsoft e Solaris da Sun. Apesar de diversas funcionalidades e melhorias desenvolvidas no Linux, o problema relacionado à falta de memória e o mecanismo existente de solucioná-lo, chamado de OOM Killer, ainda é motivo de longas discussões na comunidade do kernel Linux. A carência de pesquisas científicas relacionada ao algoritmo de seleção de processos do OOM Killer leva esta dissertação a propor um mecanismo de identificação e classificação de padrões de consumo de memória no Linux baseada no modelo de rede neural auto-organizável. A ferramenta desenvolvida nesta dissertação mostra a possibilidade de utilizar Mapas Auto-Organizáveis para classificar e identificar os padrões de consumo de memória de determinadas aplicações inseridas em contextos de casos de uso.

Abstract Pattern recognition and registration are integral elements of computer vision, which considers image patterns. This thesis presents novel blur, and combined blur and geometric invariant features for pattern recognition and registration related to images. These global or local features are based on the Fourier transform phase, and are invariant or insensitive to image blurring with a centrally symmetric point spread function which can result, for example, from linear motion or out of focus. The global features are based on the even powers of the phase-only discrete Fourier spectrum or bispectrum of an image and are invariant to centrally symmetric blur. These global features are used for object recognition and image registration. The features are extended for geometrical invariances up to similarity transformation: shift invariance is obtained using bispectrum, and rotation-scale invariance using log-polar mapping of bispectrum slices. Affine invariance can be achieved as well using rotated sets of the log-log mapped bispectrum slices. The novel invariants are shown to be more robust to additive noise than the earlier blur, and combined blur and geometric invariants based on image moments. The local features are computed using the short term Fourier transform in local windows around the points of interest. Only the lowest horizontal, vertical, and diagonal frequency coefficients are used, the phase of which is insensitive to centrally symmetric blur. The phases of these four frequency coefficients are quantized and used to form a descriptor code for the local region. When these local descriptors are used for texture classification, they are computed for every pixel, and added up to a histogram which describes the local pattern. There are no earlier textures features which have been claimed to be invariant to blur. The proposed descriptors were superior in the classification of blurred textures compared to a few non-blur invariant state of the art texture classification methods.

No description available.