Pulsars are rapidly rotating neutron stars which emit a strong beam of energy through mechanisms that are not entirely clear to physicists. These very dense stars are used by astrophysicists to study many basic physical phenomena, such as the behavior of plasmas in extremely dense environments, behavior of pulsar-black hole pairs, and tests of general relativity. Many of these tasks require information to answer the scientific questions posed by physicists. In order to provide more pulsars to study, there are several large-scale pulsar surveys underway, which are generating a huge backlog of unprocessed data. Searching for pulsars is a very labor-intensive process, currently requiring skilled people to examine and interpret plots of data output by analysis programs. An automated system for screening the plots will speed up the search for pulsars by a very large factor. Research to date on using machine learning and pattern recognition has not yielded a completely satisfactory system, as systems with the desired near 100% recall have false positive rates that are higher than desired, causing more manual labor in the classification of pulsars. This work proposed to research, identify, propose and develop methods to overcome the barriers to building an improved classification system with a false positive rate of less than 1% and a recall of near 100% that will be useful for the current and next generation of large pulsar surveys. The results show that it is possible to generate classifiers that perform as needed from the available training data. While a false positive rate of 1% was not reached, recall of over 99% was achieved with a false positive rate of less than 2%. Methods of mitigating the imbalanced training and test data were explored and found to be highly effective in enhancing classification accuracy.

Searches for pulsars during the past fifty years, have been characterised by two problems making their discovery difficult: i) an increasing volume of data to be searched, and ii) an increasing number of `candidate' pulsar detections arising from that data, requiring analysis. Whilst almost all are caused by noise or interference, these are often indistinguishable from real pulsar detections. Deciding which candidates should be studied is therefore difficult. Indeed it has become known as the `candidate selection problem'. This thesis presents an interdisciplinary study of the selection problem, with the aim of developing a new method able to mitigate it. Specifically for future pulsar surveys undertaken with the Square kilometre Array (SKA). Through a combination of critical literature evaluations, theoretical modelling exercises, and empirical investigations, the selection problem is described in-depth here for the first time. It is shown to be characterised by the dominance of Gaussian distributed noise signals, a factor that no existing selection method accounts for. It also reveals the presence of a significant trend in survey data rates, which suggest that candidate selection is transitioning from an off-line processing procedure, to an on-line, and real-time, decision making process. In response, a new real-time machine learning based method, the GH-VFDT, is introduced in this thesis. The results presented here show that a significant improvement in selection performance can be achieved using the GH-VFDT, which utilises a learning procedure optimised for data characterised by skewed class distributions. Whilst the principled development of new numerical features that maximise the separation between pulsars and Gaussian noise, have also greatly improved GH-VFDT pulsar recall. It is therefore concluded that the sub-optimal performance of existing selection systems, is due to a combination of poor feature design, insensitivity to noise, and an inability to deal with skewed class distributions.