This work is focused on the techniques for overcoming the problem of scale in the evolutionary design of the combinational multipliers. The approaches to the evolutionary design that work directly with the target solutions are not suitable for the design of the large-scale structures. An approach based on the biological principles of development has often been utilized as a non-trivial genotypephenotype mapping in the evolutionary algorithms that allows us to design scalable structures. The instruction-based developmental approach has been applied to the evolutionary design of generic circuit structures. In this work, three methods are presented for the construction of the combinational multipliers which use a ripple-carry adder for obtaining the final product.

This thesis deals with evolutionary design of the digital circuits performed by a cartesian genetic programing and optimization by a coevolution. Algorithm coevolves fitness predictors that are optimized for a population of candidate digital circuits. The thesis presents theoretical basis, especially genetic programming, coevolution in genetic programming, design of the digital circuits, and deals with possibilities of the utilization of the coevolution in the combinational circuit design. On the basis of this proposal, the application designing and optimizing logical circuits is implemented. Application functionality is verified in the five test tasks. The comparison between Cartesian genetic programming with and without coevolution is considered. Then logical circuits evolved using cartesian genetic programming with and without coevolution is compared with conventional design methods. Evolution using coevolution has reduced the number of evaluation of circuits during evolution in comparison with standard cartesian genetic programming without coevolution and in some cases is found solution with better parameters (i.e. less logical gates or less delay).

Under the Basel accords for measuring regulatory capital requirements, the set of credit risk parameters probability of default (PD), exposure at default (EAD) and loss given default (LGD) are measured with own estimates by the internal rating based approach. The estimated parameters are also the foundation of understanding the actual risk in a banks credit portfolio. The predictive performance of such models are therefore interesting to examine. The credit risk parameter LGD has been seen to give low performance for predictive models and LGD values are generally hard to estimate. The main purpose of this thesis is to analyse the predictive performance of a multi-gene genetic programming approach to symbolic regression compared to three benchmark regression models. The goal of multi-gene symbolic regression is to estimate the underlying relationship in the data through a linear combination of a set of generated mathematical expressions. The benchmark models are Logit Transformed Regression, Beta Regression and Regression Tree. All benchmark models are frequently used in the area. The data used to compare the models is a set of randomly selected, de-identified loans from the portfolios of underlying U.S. residential mortgage-backed securities retrieved from International Finance Research. The conclusion from implementing and comparing the models is that, the credit risk parameter LGD is continued difficult to estimated, the symbolic regression approach did not yield a better predictive ability than the benchmark models and it did not seem to find the underlying relationship in the data. The benchmark models are more user-friendly with easier implementation and they all requires less calculation complexity than symbolic regression.

Monitoring the heterogeneous aquatic environment such as the Stormwater Treatment Areas (STAs) located at the northeast of the Everglades is extremely important in understanding the land processes of the constructed wetland in its capacity to remove nutrient. Direct monitoring and measurements of ecosystem evolution and changing velocities at every single part of the STA are not always feasible. Integrated remote sensing, monitoring, and modeling technique can be a state-of-the-art tool to estimate the spatial and temporal distributions of flow velocity regimes and ecological functioning in such dynamic aquatic environments. In this presentation, comparison between four computational intelligence models including Extreme Learning Machine (ELM), Genetic Programming (GP) and Artificial Neural Network (ANN) models were organized to holistically assess the flow velocity and direction as well as ecosystem states within a vegetative wetland area. First the local sensor network was established using Acoustic Doppler Velocimeter (ADV). Utilizing the local sensor data along with the help of external driving forces parameters, trained models of ELM, GP and ANN were developed, calibrated, validated, and compared to select the best computational capacity of velocity prediction over time. Besides, seasonal images collected by French satellite Pleiades have been analyzed to address the seasonality effect of plant species evolution and biomass changes in the constructed wetland. The key finding of this research is to characterize the interactions between geophysical and geochemical processes in this wetland system based on ground-based monitoring sensors and satellite images to discover insight of hydraulic residence time, plant species variation, and water quality and improve the overall understanding of possible nutrient removal in this constructed wetland.

Reduction of crash occurrence on the various roadway locations (mid-block segments; signalized intersections; un-signalized intersections) and the mitigation of injury severity in the event of a crash are the major concerns of transportation safety engineers. Multi lane arterial roadways (excluding freeways and expressways) account for forty-three percent of fatal crashes in the state of Florida. Significant contributing causes fall under the broad categories of aggressive driver behavior; adverse weather and environmental conditions; and roadway geometric and traffic factors. The objective of this research was the implementation of innovative, state-of-the-art analytical methods to identify the contributing factors for crashes and injury severity. Advances in computational methods render the use of modern statistical and machine learning algorithms. Even though most of the contributing factors are known a-priori, advanced methods unearth changing trends. Heuristic evolutionary processes such as genetic programming; sophisticated data mining methods like conditional inference tree; and mathematical treatments in the form of sensitivity analyses outline the major contributions in this research. Application of traditional statistical methods like simultaneous ordered probit models, identification and resolution of crash data problems are also key aspects of this study. In order to eliminate the use of unrealistic uniform intersection influence radius of 250 ft, heuristic rules were developed for assigning crashes to roadway segments, signalized intersection and access points using parameters, such as 'site location', 'traffic control' and node information. Use of Conditional Inference Forest instead of Classification and Regression Tree to identify variables of significance for injury severity analysis removed the bias towards the selection of continuous variable or variables with large number of categories. For the injury severity analysis of crashes on highways, the corridors were clustered into four optimum groups. The optimum number of clusters was found using Partitioning around Medoids algorithm. Concepts of evolutionary biology like crossover and mutation were implemented to develop models for classification and regression analyses based on the highest hit rate and minimum error rate, respectively. Low crossover rate and higher mutation reduces the chances of genetic drift and brings in novelty to the model development process. Annual daily traffic; friction coefficient of pavements; on-street parking; curbed medians; surface and shoulder widths; alcohol / drug usage are some of the significant factors that played a role in both crash occurrence and injury severities. Relative sensitivity analyses were used to identify the effect of continuous variables on the variation of crash counts. This study improved the understanding of the significant factors that could play an important role in designing better safety countermeasures on multi lane highways, and hence enhance their safety by reducing the frequency of crashes and severity of injuries. Educating young people about the abuses of alcohol and drugs specifically at high schools and colleges could potentially lead to lower driver aggression. Removal of on-street parking from high speed arterials unilaterally could result in likely drop in the number of crashes. Widening of shoulders could give greater maneuvering space for the drivers. Improving pavement conditions for better friction coefficient will lead to improved crash recovery. Addition of lanes to alleviate problems arising out of increased ADT and restriction of trucks to the slower right lanes on the highways would not only reduce the crash occurrences but also resulted in lower injury severity levels.

One of the main purposes of the fields of Robotics and Artificial Intelligence is to develop solutions that can autonomously solve problems. An important part of this is synthesising behaviours of robots. Behaviour Trees are a tree structure that enables combining existing lower level behaviours into a high level behaviour through task switching. However, designing appropriate Behaviour Trees can be prohibitive due to time and knowledge requirements. One way of automating the creation of Behaviour Trees is through Genetic Programming, which evolves solutions through mutations and combinations akin to biological evolution. This Masters thesis explores how Genetic Programming can be used to generate Behaviour Trees in an automatic fashion. More specifically, whether so-called Quality Diversity can be used to improve the search. Quality Diversity describes a field of algorithms that combine both performance and novelty of behaviour to evaluate solutions. By including a novelty aspect the search space is more thoroughly explored, and deceptive local optima may be more easily avoided. In this thesis three Quality Diversity algorithms are implemented and evaluated in different settings: Novelty Search, Novelty Search with Local Competition, and Multi-dimensional Archive of Phenotypic Elites. The results show that Quality Diversity has potential to both increase the speed at which solutions are found and decrease the likelihood of premature convergence due to local optima. However, we also find that care must be taken in how behaviours are defined, and how some common techniques of Genetic Programming need to be adapted for Quality Diversity algorithms. / Ett av huvudsyftena med robotik och artificiell intelligens är att skapa system som självständigt kan lösa problem. En viktig del av detta är att skapa robotars beteenden. Beteendeträd är en trädstruktur som gör det möjligt att kombinera befintliga beteenden på lägre nivå till ett beteende på hög nivå. Att utforma lämpliga beteendeträd kan dock kräva både mycket tid och kunskap. Ett sätt att automatisera skapandet av beteendeträd är genom genetisk programmering, som utvecklar lösningar genom mutationer och kombinationer i likhet med biologisk evolution. Detta examensarbete undersöker hur genetisk programmering kan användas för att automatiskt generera beteendeträd. Mer specifikt undersöks om kvalitetsmångfald (Quality Diversity) kan användas för att förbättra sökningen. Kvalitetsmångfald beskriver en familj av algoritmer som kombinerar både prestanda och innovation i en lösnings beteende för att utvärdera lösningar. Genom att inkludera en innovationsaspekt blir sökområdet mer noggrant utforskat och vilseledande lokala optima kan lättare undvikas. I detta examensarbete implementeras och utvärderas tre kvalitetsmångfaldsalgoritmer i olika miljöer: Novelty Search, Novelty Search with Local Competition, och Multi-dimensional Archive of Phenotypic Elites. Resultaten visar att kvalitetsmångfald har potential att både öka hastigheten med vilken lösningar hittas och minska sannolikheten för tidig konvergens på grund av lokala optima. Vi konstaterar dock också att man måste vara försiktig med hur beteenden definieras och hur vissa vanliga tekniker för genetisk programmering måste anpassas för algoritmer med kvalitetsmångfald. / Eén van de belangrijkste doelstellingen van het vakgebied Robotica en Kunstmatige Intelligentie is het ontwikkelen van oplossingen die autonoom problemen kunnen oplossen. Een belangrijk onderdeel hiervan is het synthetiseren van gedragingen van robots. Gedragsbomen zijn een boomstructuur waarmee bestaande gedragingen van een lager niveau kunnen worden gecombineerd tot een gedrag van een hoger niveau door middel van taakwisseling. Het ontwerpen van geschikte gedragsbomen is echter soms niet haalbaar haalbaar vanwege de benodigde tijd en kennis. Een manier om het maken van gedragsbomen te automatiseren is door middel van genetisch programmeren, dat oplossingen ontwikkelt door mutaties en combinaties, vergelijkbaar met biologische evolutie. Deze masterproef onderzoekt hoe genetische programmering kan worden gebruikt om automatisch gedragsbomen te genereren. Meer specifiek of kwaliteitsdiversiteit (Quality Diversity) kan worden gebruikt om het zoeken te verbeteren. Kwaliteitsdiversiteit beschrijft een gebied van algoritmen die zowel prestaties als nieuwheid van een oplossingsgedrag combineren om oplossingen te evalueren. Door een nieuwheidsaspect te introduceren wordt de zoekruimte grondiger verkend en kunnen bedrieglijke lokale optima gemakkelijker worden vermeden. In deze masterproef worden drie algoritmen voor kwaliteitsdiversiteit toegepast en geëvalueerd in verschillende omgevingen: Novelty Search, Novelty Search with Local Competition en Multi-dimensional Archive of Phenotypic Elites. De resultaten tonen aan dat kwaliteitsdiversiteit het potentieel heeft om zowel de snelheid waarmee oplossingen worden gevonden te verhogen als de kans op voortijdige convergentie als gevolg van lokale optima te verminderen. Wij stellen echter ook vast dat zorgvuldigheid geboden is bij de definitie van gedragingen en dat sommige gebruikelijke technieken van genetisch programmeren moeten worden aangepast voor algoritmen met kwaliteitsdiversiteit.

Infrastructure deterioration has been attributed to insufficient maintenance budgets, lacking restoration strategies, deficient deterioration prediction techniques, and changing climatic conditions. Considering that the latter adds more challenges to the former, there has been a growing demand to develop and implement climate-informed infrastructure asset management strategies. However, quantifying the impact of the spatiotemporally varying climate metrics on infrastructure systems poses a serious challenge due to the associated complexities and relevant modelling uncertainties. As such, in lieu of complex physics-based simulations, the current study proposes a glass box data-driven framework for predicting infrastructure climate induced deterioration rates. The framework harnesses evolutionary computing, and specifically multigene genetic programming, to develop closed-form expressions that link infrastructure characteristics to relevant spatiotemporal climate indices and predict infrastructure deterioration rates. The framework consists of four steps: 1) data collection and preparation; 2) input integration; 3) feature selection; and 4) model development and result interpretation. To numerically demonstrate its utility, the proposed framework was applied to develop deterioration rate expressions of two different classes of concrete and steel bridges in Ontario, Canada. The developed predictive models reproduced the observed deterioration rate of both bridge classes with coefficient of determination (R2) values of 0.912 and 0.924 for the training subsets and 0.817 and 0.909 for the testing subsets of the concrete and steel bridges, respectively. Attributed to its generic nature, the framework can be applied to other infrastructure systems, with available historical deterioration data, to devise relevant effective asset management strategies and infrastructure restoration standards under future climate scenarios. / Thesis / Master of Applied Science (MASc)

В рамках исследования были проанализированы данные о кредитных операциях, предоставленные коммерческими банками. Была проведена подробная предобработка и нормализация данных для подготовки их к дальнейшему анализу и использованию в моделях машинного обучения. Основной фокус работы был сосредоточен на применении двух моделей: логистической регрессии и случайного леса. Логистическая регрессия была выбрана из-за своей простоты и интерпретируемости, а случайный лес – из-за своей способности обрабатывать большие объемы данных и выявлять сложные зависимости. В ходе экспериментов было показано, что обе модели успешно справляются с задачей оценки кредитного риска. Логистическая регрессия показала хорошую производительность, быстроту и точность, что делает ее подходящей для применения в реальном времени, например, при личной подаче заявки в банке или при онлайн-заявках. Случайный лес, в свою очередь, достиг высокой точности, хотя требует больше вычислительных ресурсов. Дополнительно, в работе был использован метод генетического программирования для создания новых признаков на основе исходных данных. Этот подход позволил значительно улучшить производительность модели и повысить ее точность. Хотя не все созданные признаки вошли в топ-5 наиболее важных, генетическое программирование оказалось эффективным способом генерации признаков, что имеет важное значение в области оценки кредитного риска. / The study analyzed data on credit transactions provided by commercial banks. Detailed pre-processing and normalization of the data was carried out to prepare it for further analysis and use in machine learning models. The main focus of the work was on the use of two models: logistic regression and random forest. Logistic regression was chosen for its simplicity and interpretability, and random forest for its ability to handle large amounts of data and identify complex relationships. During the experiments, it was shown that both models successfully cope with the task of assessing credit risk. Logistic regression has demonstrated good performance, speed, and accuracy, making it suitable for real-time applications such as in-person applications at a bank or online applications. Random forest, in turn, has achieved high accuracy, although it requires more computing resources. Additionally, the work used the genetic programming method to create new traits based on the original data. This approach significantly improved the model's performance and accuracy. Although not all of the features generated were in the top 5 most important, genetic programming has proven to be an effective way to generate features, which has important implications in the field of credit risk assessment.

Behavior Trees (BTs) are a reactive policy representation that has gained popularity in recent years, especially in the robotics domain. Among the learning methods for BTs, Genetic Programming (GP) is an effective method for learning a good BT. One drawback of GP is that it is likely to get stuck in local minima. In this project, we focus on studying both the existing methods and new directions to avoid local minima and improve the efficiency of learning BT with GP. The methods studied in the project are the grid search, the Bayesian Optimization (BO), the Distributed Island Model (DIM) and the dynamic selection pressure. We performed the experiments with four different benchmark applications implemented with high-level state machines. The changes related to fitness values, diversity, and origin throughout the learning processes were collected and analyzed as part of the quantitative analysis. Some generated BTs were selected for the qualitative analysis to provide insights into the local minima and individuals with ideal performance. Based on our experiments, we conclude that learning BTs with GP can benefit from a fitness function that is sensitive to the performance differences of the individuals. The effect of methods including the DIM and the dynamic selection pressure depends on both the applications and the settings. We recommend the grid search method for hyperparameter searching and the DIM for accelerating the learning process from distributed computing. / BTs är en reaktiv policy-representation som har ökat i popularitet de senaste åren, särskilt inom robotik. Bland inlärningsmetoderna för BTs är GP en effektiv metod för att generera bra BT. En nackdel med GP är att den lätt fastnar i lokala minima. I det här projektet fokuserar vi på att studera på existerande metoder och nya sätt att undvika lokala minima och öka inlärningseffektiviteten för BT med GP. Metoderna som studerats i projektet är grid search, BO, DIM och dynamic selection pressure. Vi genomförde experiment med fyra olika benchmarkapplikationer som implementerats med högnivå-tillståndsmaskiner. Ändringar i fitnessvärden, mångfald och källa till ändringen genom inlärningsprocessen samlades in och analyserades genom kvantitativ analys. Några genererade BTs valdes ut för kvalitativ analys för att ge insikter i de lokala minimumen och vilka individer som ger ideal prestanda. Baserat på våra experiment konkluderar vi att inlärning av BTs med GP kan tjäna på en bra fitnessfunktion som är känslig för prestandaskillnader mellan invidider. Effekten av metoderna DIM och dynamic selection pressure beror på applikationen och inställningarna. Vi rekommenderar grid search för hyperparametersökning och DIM för att accelerera inlärningen från distribuerade system.

Doctor of Philosophy / Department of Electrical and Computer Engineering / Sanjoy Das / Stochastic algorithms are widely used in various modeling and optimization problems. Evolutionary algorithms are one class of population-based stochastic approaches that are inspired from Darwinian evolutionary theory. A population of candidate solutions is initialized at the first generation of the algorithm. Two variation operators, crossover and mutation, that mimic the real world evolutionary process, are applied on the population to produce new solutions from old ones. Selection based on the concept of survival of the fittest is used to preserve parent solutions for next generation. Examples of such algorithms include genetic algorithm (GA) and genetic programming (GP). Nevertheless, other stochastic algorithms may be inspired from animals’ behavior such as particle swarm optimization (PSO), which imitates the cooperation of a flock of birds. In addition, stochastic algorithms are able to address multi-objective optimization problems by using the concept of dominance. Accordingly, a set of solutions that do not dominate each other will be obtained, instead of just one best solution. This thesis proposes a multi-objective GP-PSO hybrid algorithm to recover gene regulatory network models that take environmental data as stimulus input. The algorithm infers a model based on both phenotypic and gene expression data. The proposed approach is able to simultaneously infer network structures and estimate their associated parameters, instead of doing one or the other iteratively as other algorithms need to. In addition, a non-dominated sorting approach and an adaptive histogram method based on the hypergrid strategy are adopted to address ‘convergence’ and ‘diversity’ issues in multi-objective optimization. Gene network models obtained from the proposed algorithm are compared to a synthetic network, which mimics key features of Arabidopsis flowering control system, visually and numerically. Data predicted by the model are compared to synthetic data, to verify that they are able to closely approximate the available phenotypic and gene expression data. At the end of this thesis, a novel breeding strategy, termed network assisted selection, is proposed as an extension of our hybrid approach and application of obtained models for plant breeding. Breeding simulations based on network assisted selection are compared to one common breeding strategy, marker assisted selection. The results show that NAS is better both in terms of breeding speed and final phenotypic level.