No description available.

This thesis presents an implementation of a learning classifier system which learns good database schema. The system is implemented in Java using the NetBeans development environment, which provides a good control for the GUI components. The system contains four components: a user interface, a rule and message system, an apportionment of credit system, and genetic algorithms. The input of the system is a set of simple database schemas and the objective for the classifier system is to keep the good database schemas which are represented by classifiers. The learning classifier system is given some basic knowledge about database concepts or rules. The result showed that the system could decrease the bad schemas and keep the good ones.

Access technologies often deal with the classification of several physiological and biomechanical signals. In most previous studies involving access technologies, a single classifier has been trained. Despite reported success of these single classifiers, classification accuracies are often below clinically viable levels. One approach to improve upon the performance of these classifiers is to utilize the state of- the-art multiple classifier systems (MCS). Because MCS invoke more than one classifier, more information can be exploited from the signals, potentially leading to higher classification performance than that achievable with single classifiers. Moreover, by decreasing the feature space dimensionality of each classifier, the speed of the system can be increased. MCSs may combine classifiers on three levels: abstract, rank, or measurement level. Among them, abstract-level MCSs have been the most widely applied in the literature given the flexibility of the abstract level output, i.e., class labels may be derived from any type of classifier and outputs from multiple classifiers, each designed within a different context, can be easily combined. In this thesis, we develop two new abstract-level MCSs based on "reputation" values of individual classifiers: the static reputation-based algorithm (SRB) and the dynamic reputation-based algorithm (DRB). In SRB, each individual classifier is applied to a “validation set”, which is disjoint from training and test sets, to estimate its reputation value. Then, each individual classifier is assigned a weight proportional to its reputation value. Finally, the total decision of the classification system is computed using Bayes rule. We have applied this method to the problem of dysphagia detection in adults with neurogenic swallowing difficulties. The aim was to discriminate between safe and unsafe swallows. The weighted classification accuracy exceeded 85% and, because of its high sensitivity, the SRB approach was deemed suitable for screening purposes. In the next step of this dissertation, I analyzed the SRB algorithm mathematically and examined its asymptotic behavior. Specifically, I contrasted the SRB performance against that of majority voting, the benchmark abstract-level MCS, in the presence of different types of noise. In the second phase of this thesis, I exploited the idea of the Dirichlet reputation system to develop a new MCS method, the dynamic reputation-based algorithm, which is suitable for the classification of non-stationary signals. In this method, the reputation of each classifier is updated dynamically whenever a new sample is classified. At any point in time, a classifier’s reputation reflects the classifier’s performance on both the validation and the test sets. Therefore, the effect of random high-performance of weak classifiers is appropriately moderated and likewise, the effect of a poorly performing individual classifier is mitigated as its reputation value, and hence overall influence on the final decision is diminished. We applied DRB to the challenging problem of discerning physiological responses from nonverbal youth with severe disabilities. The promising experimental results encourage further development of reputation-based multi-classifier systems in the domain of access technology research.

Access technologies often deal with the classification of several physiological and biomechanical signals. In most previous studies involving access technologies, a single classifier has been trained. Despite reported success of these single classifiers, classification accuracies are often below clinically viable levels. One approach to improve upon the performance of these classifiers is to utilize the state of- the-art multiple classifier systems (MCS). Because MCS invoke more than one classifier, more information can be exploited from the signals, potentially leading to higher classification performance than that achievable with single classifiers. Moreover, by decreasing the feature space dimensionality of each classifier, the speed of the system can be increased. MCSs may combine classifiers on three levels: abstract, rank, or measurement level. Among them, abstract-level MCSs have been the most widely applied in the literature given the flexibility of the abstract level output, i.e., class labels may be derived from any type of classifier and outputs from multiple classifiers, each designed within a different context, can be easily combined. In this thesis, we develop two new abstract-level MCSs based on "reputation" values of individual classifiers: the static reputation-based algorithm (SRB) and the dynamic reputation-based algorithm (DRB). In SRB, each individual classifier is applied to a “validation set”, which is disjoint from training and test sets, to estimate its reputation value. Then, each individual classifier is assigned a weight proportional to its reputation value. Finally, the total decision of the classification system is computed using Bayes rule. We have applied this method to the problem of dysphagia detection in adults with neurogenic swallowing difficulties. The aim was to discriminate between safe and unsafe swallows. The weighted classification accuracy exceeded 85% and, because of its high sensitivity, the SRB approach was deemed suitable for screening purposes. In the next step of this dissertation, I analyzed the SRB algorithm mathematically and examined its asymptotic behavior. Specifically, I contrasted the SRB performance against that of majority voting, the benchmark abstract-level MCS, in the presence of different types of noise. In the second phase of this thesis, I exploited the idea of the Dirichlet reputation system to develop a new MCS method, the dynamic reputation-based algorithm, which is suitable for the classification of non-stationary signals. In this method, the reputation of each classifier is updated dynamically whenever a new sample is classified. At any point in time, a classifier’s reputation reflects the classifier’s performance on both the validation and the test sets. Therefore, the effect of random high-performance of weak classifiers is appropriately moderated and likewise, the effect of a poorly performing individual classifier is mitigated as its reputation value, and hence overall influence on the final decision is diminished. We applied DRB to the challenging problem of discerning physiological responses from nonverbal youth with severe disabilities. The promising experimental results encourage further development of reputation-based multi-classifier systems in the domain of access technology research.

To solve the challenging pattern classification problem, machine learning researchers have extensively studied Multiple Classifier Systems (MCSs). The motivations for combining classifiers are found in the literature from the statistical, computational and representational perspectives. Although the results of classifier combination does not always outperform the best individual classifier in the ensemble, empirical studies have demonstrated its superiority for various applications. A number of viable methods to design MCSs have been developed including bagging, adaboost, rotation forest, and random subspace. They have been successfully applied to solve various tasks. Currently, most of the research is being conducted on the behavior patterns of the base classifiers in the ensemble. However, a discussion from the learning point of view may provide insights into the robust design of MCSs. In this thesis, Generalized Exhaustive Search and Aggregation (GESA) method is developed for this objective. Robust performance is achieved using GESA by dynamically adjusting the trade-off between fitting the training data adequately and preventing the overfitting problem. Besides its learning algorithm, GESA is also distinguished from traditional designs by its architecture and level of decision-making. GESA generates a collection of ensembles and dynamically selects the most appropriate ensemble for decision-making at the local level. Although GESA provides a good improvement over traditional approaches, it is not very data-adaptive. A data- adaptive design of MCSs demands that the system can adaptively select representations and classifiers to generate effective decisions for aggregation. Another weakness of GESA is its high computation cost which prevents it from being scaled to large ensembles. Generalized Adaptive Ensemble Generation and Aggregation (GAEGA) is an extension of GESA to overcome these two difficulties. GAEGA employs a greedy algorithm to adaptively select the most effective representations and classifiers while excluding the noise ones as much as possible. Consequently, GAEGA can generate fewer ensembles and significantly reduce the computation cost. Bootstrapped Adaptive Ensemble Generation and Aggregation (BAEGA) is another extension of GESA, which is similar with GAEGA in the ensemble generation and decision aggregation. BAEGA adopts a different data manipulation strategy to improve the diversity of the generated ensembles and utilize the information in the data more effectively. As a specific application, the classification of time series data is chosen for the research reported in this thesis. This type of data contains dynamic information and proves to be more complex than others. Multiple Input Representation-Adaptive Ensemble Generation and Aggregation (MIR-AEGA) is derived from GAEGA for the classification of time series data. MIR-AEGA involves some novel representation methods that proved to be effective for time series data. All the proposed methods including GESA, GAEGA, MIR-AEGA, and BAEGA are tested on simulated and benchmark data sets from popular data repositories. The experimental results confirm that the newly developed methods are effective and efficient.

Multiple classifier system has shown to be an effective technique for classification. The success of multiple classifiers does not entirely depend on the base classifiers and/or the aggregation technique. Other parameters, such as training data, feature attributes, and correlation among the base classifiers may also contribute to the success of multiple classifiers. In addition, interaction of these parameters with each other may have an impact on multiple classifiers performance. In the present study, we intended to examine some of these interactions and investigate further the effects of these interactions on the performance of classifier ensembles. The proposed research introduces a different direction in the field of multiple classifiers systems. We attempt to understand and compare ensemble methods from the cooperation perspective. In this thesis, we narrowed down our focus on cooperation at training level. We first developed measures to estimate the degree and type of cooperation among training data partitions. These evaluation measures enabled us to evaluate the diversity and correlation among a set of disjoint and overlapped partitions. With the aid of properly selected measures and training information, we proposed two new data partitioning approaches: Cluster, De-cluster, and Selection (CDS) and Cooperative Cluster, De-cluster, and Selection (CO-CDS). In the end, a comprehensive comparative study was conducted where we compared our proposed training approaches with several other approaches in terms of robustness of their usage, resultant classification accuracy and classification stability. Experimental assessment of CDS and CO-CDS training approaches validates their robustness as compared to other training approaches. In addition, this study suggests that: 1) cooperation is generally beneficial and 2) classifier ensembles that cooperate through sharing information have higher generalization ability compared to the ones that do not share training information.

To solve the challenging pattern classification problem, machine learning researchers have extensively studied Multiple Classifier Systems (MCSs). The motivations for combining classifiers are found in the literature from the statistical, computational and representational perspectives. Although the results of classifier combination does not always outperform the best individual classifier in the ensemble, empirical studies have demonstrated its superiority for various applications. A number of viable methods to design MCSs have been developed including bagging, adaboost, rotation forest, and random subspace. They have been successfully applied to solve various tasks. Currently, most of the research is being conducted on the behavior patterns of the base classifiers in the ensemble. However, a discussion from the learning point of view may provide insights into the robust design of MCSs. In this thesis, Generalized Exhaustive Search and Aggregation (GESA) method is developed for this objective. Robust performance is achieved using GESA by dynamically adjusting the trade-off between fitting the training data adequately and preventing the overfitting problem. Besides its learning algorithm, GESA is also distinguished from traditional designs by its architecture and level of decision-making. GESA generates a collection of ensembles and dynamically selects the most appropriate ensemble for decision-making at the local level. Although GESA provides a good improvement over traditional approaches, it is not very data-adaptive. A data- adaptive design of MCSs demands that the system can adaptively select representations and classifiers to generate effective decisions for aggregation. Another weakness of GESA is its high computation cost which prevents it from being scaled to large ensembles. Generalized Adaptive Ensemble Generation and Aggregation (GAEGA) is an extension of GESA to overcome these two difficulties. GAEGA employs a greedy algorithm to adaptively select the most effective representations and classifiers while excluding the noise ones as much as possible. Consequently, GAEGA can generate fewer ensembles and significantly reduce the computation cost. Bootstrapped Adaptive Ensemble Generation and Aggregation (BAEGA) is another extension of GESA, which is similar with GAEGA in the ensemble generation and decision aggregation. BAEGA adopts a different data manipulation strategy to improve the diversity of the generated ensembles and utilize the information in the data more effectively. As a specific application, the classification of time series data is chosen for the research reported in this thesis. This type of data contains dynamic information and proves to be more complex than others. Multiple Input Representation-Adaptive Ensemble Generation and Aggregation (MIR-AEGA) is derived from GAEGA for the classification of time series data. MIR-AEGA involves some novel representation methods that proved to be effective for time series data. All the proposed methods including GESA, GAEGA, MIR-AEGA, and BAEGA are tested on simulated and benchmark data sets from popular data repositories. The experimental results confirm that the newly developed methods are effective and efficient.

The growing problem of spam has generated a need for reliable anti-spam filters. There are many filtering techniques along with machine learning and data miming used to reduce the amount of spam. Such algorithms can achieve very high accuracy but with some amount of false positive tradeoff. Generally false positives are prohibitively expensive in the real world. Much work has been done to improve specific algorithms for the task of detecting spam, but less work has been report on leveraging multiple algorithms in email analysis. This study presents an alliance-based approach to classify, discovery and exchange interesting information on spam. Furthermore, the spam filter in this study is build base on the mixture of rough set theory (RST), genetic algorithm (GA) and XCS classifier system. RST has the ability to process imprecise and incomplete data such as spam. GA can speed up the rate of finding the optimal solution (i.e. the rules used to block spam). The reinforcement learning of XCS is a good mechanism to suggest the appropriate classification for the email. The results of spam filtering by alliance-based approach are evaluated by several statistical methods and the performance is great. Two main conclusions can be drawn from this study: (1) the rules exchanged from other mail servers indeed help the filter blocking more spam than before. (2) a combination of algorithms improves both accuracy and reducing false positives for the problem of spam detection.

Multiple classifier system has shown to be an effective technique for classification. The success of multiple classifiers does not entirely depend on the base classifiers and/or the aggregation technique. Other parameters, such as training data, feature attributes, and correlation among the base classifiers may also contribute to the success of multiple classifiers. In addition, interaction of these parameters with each other may have an impact on multiple classifiers performance. In the present study, we intended to examine some of these interactions and investigate further the effects of these interactions on the performance of classifier ensembles. The proposed research introduces a different direction in the field of multiple classifiers systems. We attempt to understand and compare ensemble methods from the cooperation perspective. In this thesis, we narrowed down our focus on cooperation at training level. We first developed measures to estimate the degree and type of cooperation among training data partitions. These evaluation measures enabled us to evaluate the diversity and correlation among a set of disjoint and overlapped partitions. With the aid of properly selected measures and training information, we proposed two new data partitioning approaches: Cluster, De-cluster, and Selection (CDS) and Cooperative Cluster, De-cluster, and Selection (CO-CDS). In the end, a comprehensive comparative study was conducted where we compared our proposed training approaches with several other approaches in terms of robustness of their usage, resultant classification accuracy and classification stability. Experimental assessment of CDS and CO-CDS training approaches validates their robustness as compared to other training approaches. In addition, this study suggests that: 1) cooperation is generally beneficial and 2) classifier ensembles that cooperate through sharing information have higher generalization ability compared to the ones that do not share training information.

The topic of this thesis is knowledge discovery and artificial intelligence based knowledge discovery algorithms. The knowledge discovery process and associated problems are discussed, followed by an overview of three classes of artificial intelligence based knowledge discovery algorithms. Typical representatives of each of these classes are presented and discussed in greater detail. Then a new knowledge discovery algorithm, called Hybrid Classifier System (HCS), is presented. The guiding concept behind the new algorithm was simplicity. The new knowledge discovery algorithm is loosely based on schemata theory. It is evaluated against one of the discussed algorithms from each class, namely: CN2; C4.5, BRAINNE and BGP. Results are discussed and compared. A comparison was done using a benchmark of classification problems. These results show that the new knowledge discovery algorithm performs satisfactory, yielding accurate, crisp rule sets. Probably the main strength of the HCS algorithm is its simplicity, so it can be the foundation for many possible future extensions. Some of the possible extensions of the new proposed algorithm are suggested in the final part of this thesis. / Dissertation (MSc)--University of Pretoria, 2007. / Computer Science / unrestricted