This thesis examines a conversion of a solution produced by geometric semantic genetic programming (GSGP) to an instantion of cartesian genetic programming (CGP). GSGP has proven its quality to create complex mathematical models; however, the size of these models can get problematically large. CGP, on the other hand, is able to reduce the size of given models. This thesis combinated these methods to create a subtree CGP (SCGP). The SCGP uses an output of GSGP as an input and the evolution is performed using the CGP. Experiments performed on four pharmacokinetic tasks have shown that the SCGP is able to reduce the solution size in every case. Overfitting was detected in one out of four test problems.

This thesis deals with image classification based on genetic programming and coevolution. Genetic programming algorithms make generating executable structures possible, which allows us to design solutions in form of programs. Using coevolution with the fitness prediction lowers the amount of time consumed by fitness evaluation and, therefore, also the execution time. The thesis describes a theoretical background of evolutionary algorithms and, in particular, cartesian genetic programming. We also describe coevolutionary algorithms properties and especially the proposed method for the image classifier evolution using coevolution of fitness predictors, where the objective is to find a good compromise between the classification accuracy, design time and classifier complexity. A part of the thesis is implementation of the proposed method, conducting the experiments and comparison of obtained results with other methods.

The aim of this work is to design and implement a method for the evolutionary design of neural networks with generative encoding. The proposed method is based on J. F. Miller's approach and uses a brain model that is gradually developed and which allows extraction of traditional neural networks. The development of the brain is controlled by programs created using cartesian genetic programming. The project was implemented in Python with the use of Numpy library. Experiments have shown that the proposed method is able to construct neural networks that achieve over 90 % accuracy on smaller datasets. The method is also able to develop neural networks capable of solving multiple problems at once while slightly reducing accuracy.

Symbolic regression is the problem of identifying the mathematic description of a hidden system from experimental data. Symbolic regression is closely related to general machine learning. This work deals with symbolic regression and its solution based on the principle of genetic programming and coevolution. Genetic programming is the evolution based machine learning method, which automaticaly generates whole programs in the given programming language. Coevolution of fitness predictors is the optimalization method of the fitness modelling that reduces the fitness evaluation cost and frequency, while maintainig evolutionary progress. This work deals with concept and implementation of the solution of symbolic regression using coevolution of fitness predictors, and its comparison to a solution without coevolution. Experiments were performed using cartesian genetic programming.

During the last years cartesian genetic programming proved to be a very perspective area of the evolutionary computing. However it has its limitations, which make its use in area of large and generic problems impossible. These limitations can be eliminated using the recent method allowing self-modification of programs in cartesian genetic programming. The purpose of this thesis is to review the development in this area done so far. Next objective is to design own solutions for solving various problems that are hardly solvable using the ordinary cartesian genetic programming. One of the problems to be considered is generating the terms of various Taylor series. Due to the fact that the solution to this problem requires generalisation, the goal is to prove that the self-modifying cartesian genetic programming scores better than classic one for this problem. Another discussed problem is using the self-modifying genetic programming for developing arbitrarily large sorting networks. In this case, the objective is to prove that self-modification brings new features to the cartesian genetic programming allowing the development of arbitrarily sized designs.

Agent-based computing has attracted much attention as a promising technique for application domains that are distributed, complex and heterogeneous. Current research on multi-agent systems (MAS) has become mature enough to be applied as a technology for solving problems in an increasingly wide range of complex applications. The main formal architectures used to describe the relationships between agents in MAS are centralised and distributed architectures. In computational complexity theory, researchers have classified the problems into the followings categories: (i) P problems, (ii) NP problems, (iii) NP-complete problems, and (iv) NP-hard problems. A method for computing the solution to NP-hard problems, using the algorithms and computational power available nowadays in reasonable time frame remains undiscovered. And unfortunately, many practical problems belong to this very class. On the other hand, it is essential that these problems are solved, and the only possibility of doing this is to use approximation techniques. Heuristic solution techniques are an alternative. A heuristic is a strategy that is powerful in general, but not absolutely guaranteed to provide the best (i.e. optimal) solutions or even find a solution. This demands adopting some optimisation techniques such as Evolutionary Algorithms (EA). This research has been undertaken to investigate the feasibility of running computationally intensive algorithms on multi-agent architectures while preserving the ability of small agents to run on small devices, including mobile devices. To achieve this, the present work proposes a new Hybrid Multi-Agent Architecture (HMAA) that generates new heuristics for solving NP-hard problems. This architecture is hybrid because it is "semi-distributed/semi-centralised" architecture where variables and constraints are distributed among small agents exactly as in distributed architectures, but when the small agents become stuck, a centralised control becomes active where the variables are transferred to a super agent, that has a central view of the whole system, and possesses much more computational power and intensive algorithms to generate new heuristics for the small agents, which find optimal solution for the specified problem. This research comes up with the followings: (1) Hybrid Multi-Agent Architecture (HMAA) that generates new heuristic for solving many NP-hard problems. (2) Two frameworks of HMAA have been implemented; search and optimisation frameworks. (3) New SMA meeting scheduling heuristic. (4) New SMA repair strategy for the scheduling process. (5) Small Agent (SMA) that is responsible for meeting scheduling has been developed. (6) “Local Search Programming” (LSP), a new concept for evolutionary approaches, has been introduced. (7) Two types of super-agent (LGP_SUA and LSP_SUA) have been implemented in the HMAA, and two SUAs (local and global optima) have been implemented for each type. (8) A prototype for HMAA has been implemented: this prototype employs the proposed meeting scheduling heuristic with the repair strategy on SMAs, and the four extensive algorithms on SUAs. The results reveal that this architecture is applicable to many different application domains because of its simplicity and efficiency. Its performance was better than many existing meeting scheduling architectures. HMAA can be modified and altered to other types of evolutionary approaches.

Management of water quality is generally based on physically-based equations or hypotheses describing the behaviour of water bodies. In recent years models built on the basis of the availability of larger amounts of collected data are gaining popularity. This modelling approach can be called data driven modelling. Observational data represent specific knowledge, whereas a hypothesis represents a generalization of this knowledge that implies and characterizes all such observational data. Traditionally deterministic numerical models have been used for predicting flow and water quality processes in inland and coastal basins. These models generally take a long time to run and cannot be used as on-line decision support tools, thereby enabling imminent threats to public health risk and flooding etc. to be predicted. In contrast, Data driven models are data intensive and there are some limitations in this approach. The extrapolation capability of data driven methods are a matter of conjecture. Furthermore, the extensive data required for building a data driven model can be time and resource consuming or for the case predicting the impact of a future development then the data is unlikely to exist. The main objective of the study was to develop an integrated approach for rapid prediction of bathing water quality in estuarine and coastal waters. Faecal Coliforms (FC) were used as a water quality indicator and two of the most popular data mining techniques, namely, Genetic Programming (GP) and Artificial Neural Networks (ANNs) were used to predict the FC levels in a pilot basin. In order to provide enough data for training and testing the neural networks, a calibrated hydrodynamic and water quality model was used to generate input data for the neural networks. A novel non-linear data analysis technique, called the Gamma Test, was used to determine the data noise level and the number of data points required for developing smooth neural network models. Details are given of the data driven models, numerical models and the Gamma Test. Details are also given of a series experiments being undertaken to test data driven model performance for a different number of input parameters and time lags. The response time of the receiving water quality to the input boundary conditions obtained from the hydrodynamic model has been shown to be a useful knowledge for developing accurate and efficient neural networks. It is known that a natural phenomenon like bacterial decay is affected by a whole host of parameters which can not be captured accurately using solely the deterministic models. Therefore, the data-driven approach has been investigated using field survey data collected in Cardiff Bay to investigate the relationship between bacterial decay and other parameters. Both of the GP and ANN models gave similar, if not better, predictions of the field data in comparison with the deterministic model, with the added benefit of almost instant prediction of the bacterial levels for this recreational water body. The models have also been investigated using idealised and controlled laboratory data for the velocity distributions along compound channel reaches with idealised rods have located on the floodplain to replicate large vegetation (such as mangrove trees).

This thesis explores Web Information Extraction (WIE) and how it has been used in decision making and to support businesses in their daily operations. The research focuses on a WIE system based on Genetic Programming (GP) with an extensible model to enhance the automatic extractor. This uses a human as a teacher to identify and extract relevant information from the semi-structured HTML webpages. Regular expressions, which have been chosen as the pattern matching tool, are automatically generated based on the training data to provide an improved grammar and lexicon. This particularly benefits the GP system which may need to extend its lexicon in the presence of new tokens in the web pages. These tokens allow the GP method to produce new extraction patterns for new requirements.

Thesis (MScEng (Process Engineering))--University of Stellenbosch, 2006. / Genetic programming (GP) is a methodology that imitates genetic algorithms, which uses mutation and replication to produce algorithms or model structures based on Darwinian survival-of-the-fittest principles. Despite its obvious po-tential in process systems engineering, GP does not appear to have gained large-scale acceptance in process engineering applications. In this thesis, therefore, the following hypothesis was considered: Genetic programming offers a competitive approach towards the automatic generation of process models from data. This was done by comparing three different GP algorithms to classification and regression trees (CART) as benchmark. Although these models could be assessed on the basis of several different criteria, the assessment was limited to the predictive power and interpretability of the models. The reason for using CART as a benchmark, was that it is well-established as a nonlinear approach to modelling, and more importantly, it can generate interpretable models in the form of IF-THEN rules. Six case studies were considered. Two of these were based on simulated data (a regression and a classification problem), while the other four were based on real-world data obtained from the process industries (three classification problems and one regression problem). In the two simulated case studies, the CART models outperformed the GP models both in terms of predictive power and interpretability. In the four real word case studies, two of the GP algorithms and CART performed equally in terms of predictive power. Mixed results were obtained as far as the interpretability of the models was concerned. The CART models always produced sets of IF-THEN rules that were in principle easy to interpret. However, when many of these rules are needed to represent the system (large trees), the tree models lose their interpretability – as was indeed the case in the majority of the case studies considered. Nonetheless, the CART models produced more interpretable structures in almost all the case studies. The exception was a case study related to the classification of hot rolled steel plates (which could have surface defects or not). In this case, the one of the GP models produced a singularly simple model, with the same predictive power as that of the classification tree. Although GP models and their construction were generally more complex than classification/regression models and did not appear to afford any particular advantages in predictive power over the classification/regression trees, they could therefore provide more concise, interpretable models than CART. For this reason, the hypothesis of the thesis should arguably be accepted, especially if a high premium is placed on the development of interpretable models.

This thesis tackles a series of problems related to the evolution of completesoftware systems both in terms of the underlying Genetic Programmingsystem and the application of that system. A new representation is presented that addresses some of the issues withother Genetic Program representations while keeping their advantages. Thiscombines the easy reproduction of the linear representation with the inheritablecharacteristics of the tree representation by using fixed-length blocks ofgenes representing single program statements. This means that each block ofgenes will always map to the same statement in the parent and child unless itis mutated, irrespective of changes to the surrounding blocks. This methodis compared to the variable length gene blocks used by other representationswith a clear improvement in the similarity between parent and child. Traditionally, fitness functions have either been created as a selection ofsample inputs with known outputs or as hand-crafted evaluation functions. Anew method of creating fitness evaluation functions is introduced that takesthe formal specification of the desired function as its basis. This approachensures that the fitness function is complete and concise. The fitness functionscreated from formal specifications are compared to simple input/outputpairs and the results show that the functions created from formal specificationsperform significantly better. A set of list evaluation and manipulation functions was evolved as anapplication of the new Genetic Program components. These functions havethe common feature that they all need to be 100% correct to be useful. Traditional Genetic Programming problems have mainly been optimizationor approximation problems. The list results are good but do highlight theproblem of scalability in that more complex functions lead to a dramaticincrease in the required evolution time. Finally, the evolution of graphical user interfaces is addressed. The representationfor the user interfaces is based on the new representation forprograms. In this case each gene block represents a component of the userinterface. The fitness of the interface is determined by comparing it to a seriesof constraints, which specify the layout, style and functionality requirements. A selection of web-based and desktop-based user interfaces were evolved. With these new approaches to Genetic Programming, the evolution ofcomplete software systems is now a realistic goal.