Neural networks are used in various fields to make predictions about the future value of a time series, or about the class membership of a given object. For the network to be effective, it needs to be trained on a set of training data combined with the expected results. Two aspects to keep in mind when considering a neural network as a solution, are the required training time and the prediction accuracy. This research compares the classification accuracy of conjugate gradient descent neural networks and Bayesian learning neural networks. Conjugate gradient descent networks are known for their short training times, but are not very consistent and results are heavily dependant on initial training conditions. Bayesian networks are slower, but much more consistent. The two types of neural networks are compared, and some attempts are made to combine their strong points in order to achieve shorter training times while maintaining a high classification accuracy. Bayesian learning outperforms the gradient descent methods by almost 1%, while the hybrid method achieves results between those of Bayesian learning and gradient descent. The drawback of the hybrid method is that there is no speed improvement above that of Bayesian learning. / Dissertation (MEng (Electronics))--University of Pretoria, 2005. / Electrical, Electronic and Computer Engineering / unrestricted

No description available.

Published Article / The paper is concerned with the problem of Bayesian decision-makers seeking consensus about the decision that should be taken from a decision space. Each decision-maker has his own utility function and it is assumed that the parameter space has two points, Θ = {θ1,θ2 }. The initial probabilities of the decision-makers for Θ can be updated by information provided by an expert. The decision-makers have an opinion about the expert and this opinion is formed by the observation of the expert's performance in the past. It is shown how the decision-makers can decide beforehand, on the basis of this opinion, whether the consultation of an expert will result in consensus.

I argue against the halfer response to the Sleeping Beauty case by presenting a new problem for halfers. When the original Sleeping Beauty case is generalized, it follows from the halfer’s key premise that Beauty must update her credence in a fair coin’s landing heads in such a way that it becomes arbitrarily close to certainty. This result is clearly absurd. I go on to argue that the halfer’s key premise must be rejected on pain of absurdity, leaving the halfer response to the original Sleeping Beauty case unsupported. I consider two ways that halfers might avoid the absurdity without giving up their key premise. Neither way succeeds. My argument lends support to the thirder response, and, in particular, to the idea that agents may be rationally compelled to update their beliefs despite not having learned any new evidence.

Bayesian analysis is definitely different from the classic statistical methods. Although, both of them use subjective ideas, it is used in the selection of models in the classic statistical methods, rather than as an explicit part in Bayesian models, which allows the combination of subjective ideas with the data collected, update the prior information and improve inferences. Drastic growth of Bayesian applications indicates it becomes more and more popular, because the advent of computational methods (e.g., MCMC) renders sophisticated analysis. In Bayesian framework, the flexibility and generality allows it to cope with very complex problems. One big obstacle in earlier Bayesian analysis is how to sample from the usually complex posterior distribution. With modern techniques and fast-developed computation capacity, we now have tools to solve this problem. We discuss Acceptance-Rejection sampling, importance sampling and then the MCMC methods. Metropolis-Hasting algorithm, as a very versatile, efficient and powerful simulation technique to construct a Markov Chain, borrows the idea from the well-known acceptance-rejection sampling to generate candidates that are either accepted or rejected, but then retains the current values when rejection takes place (1). A special case of Metropolis-Hasting algorithm is Gibbs Sampler. When dealing with high dimensional problems, Gibbs Sampler doesn’t require a decent proposal distribution. It generates the Markov Chain through univariate conditional probability distribution, which greatly simplifies problems. We illustrate the use of those approaches with examples (with R codes) to provide a thorough review. Those basic methods have variants to deal with different situations. And they are building blocks for more advanced problems. This report is not a tutorial for statistics or the software R. The author assumes that readers are familiar with basic statistical concepts and common R statements. If needed, a detailed instruction of R programming can be found in the Comprehensive R Archive Network (CRAN): http://cran.R-project.org / text

No description available.

Locally weighted regression is a non-parametric technique of regression that is capable of coping with non-stationarity of the input distribution. Online algorithms like Receptive FieldWeighted Regression and Locally Weighted Projection Regression use a sparse representation of the locally weighted model to approximate a target function, resulting in an efficient learning algorithm. However, these algorithms are fairly sensitive to parameter initializations and have multiple open learning parameters that are usually set using some insights of the problem and local heuristics. In this thesis, we attempt to alleviate these problems by using a probabilistic formulation of locally weighted regression followed by a principled Bayesian inference of the parameters. In the Randomly Varying Coefficient (RVC) model developed in this thesis, locally weighted regression is set up as an ensemble of regression experts that provide a local linear approximation to the target function. We train the individual experts independently and then combine their predictions using a Product of Experts formalism. Independent training of experts allows us to adapt the complexity of the regression model dynamically while learning in an online fashion. The local experts themselves are modeled using a hierarchical Bayesian probability distribution with Variational Bayesian Expectation Maximization steps to learn the posterior distributions over the parameters. The Bayesian modeling of the local experts leads to an inference procedure that is fairly insensitive to parameter initializations and avoids problems like overfitting. We further exploit the Bayesian inference procedure to derive efficient online update rules for the parameters. Learning in the regression setting is also extended to handle a classification task by making use of a logistic regression to model discrete class labels. The main contribution of the thesis is a spatially localised online learning algorithm set up in a probabilistic framework with principled Bayesian inference rule for the parameters of the model that learns local models completely independent of each other, uses only local information and adapts the local model complexity in a data driven fashion. This thesis, for the first time, brings together the computational efficiency and the adaptability of ‘non-competitive’ locally weighted learning schemes and the modelling guarantees of the Bayesian formulation.

No description available.

No description available.

In this thesis, a Bayes linear methodology for the adjustment of covariance matrices is presented and discussed. A geometric framework for quantifying uncertainties about covariance matrices is set up, and an inner-product for spaces of random matrices is motivated and constructed. The inner-product on this space captures aspects of belief about the relationships between covariance matrices of interest, providing a structure rich enough to adjust beliefs about unknown matrices in the light of data such as sample covariance matrices, exploiting second-order exchangeability and related specifications to obtain representations allowing analysis. Adjustment is associated with orthogonal projection, and illustrated by examples for some common problems. The difficulties of adjusting the covariance matrices underlying exchangeable random vectors is tackled and discussed. Learning about the covariance matrices associated with multivariate time series dynamic linear models is shown to be amenable to a similar approach. Diagnostics for matrix adjustments are also discussed.