ITC/USA 2011 Conference Proceedings / The Forty-Seventh Annual International Telemetering Conference and Technical Exhibition / October 24-27, 2011 / Bally's Las Vegas, Las Vegas, Nevada / This paper describes a service-based Intelligent Agent (IA) approach for machine learning and data mining of distributed heterogeneous data streams. We focus on an open architecture framework that enables the programmer/analyst to build an IA suite for mining, examining and evaluating heterogeneous data for semantic representations, while iteratively building the probabilistic model in real-time to improve predictability. The Framework facilitates model development and evaluation while delivering the capability to tune machine learning algorithms and models to deliver increasingly favorable scores prior to production deployment. The IA Framework focuses on open standard interoperability, simplifying integration into existing environments.

Non-linearities and dynamic interactions between state variables are characteristic of complex social systems and processes. In this thesis, we present a new methodology to model these non-linearities and interactions from the large panel datasets available for some of these systems. We build macro-level statistical models that can verify theoretical predictions, and use polynomial basis functions so that each term in the model represents a specific mechanism. This bridges the existing gap between macro-level theories supported by statistical models and micro-level mechanistic models supported by behavioural evidence. We apply this methodology to two important problems in the social sciences, the demographic transition and the transition to democracy. The demographic transition is an important problem for economists and development scientists. Research has shown that economic growth reduces mortality and fertility rates, which reduction in turn results in faster economic growth. We build a non-linear dynamic model and show how this data-driven model extends existing mechanistic models. We also show policy applications for our models, especially in setting development targets for the Millennium Development Goals or the Sustainable Development Goals. The transition to democracy is an important problem for political scientists and sociologists. Research has shown that economic growth and overall human development transforms socio-cultural values and drives political institutions towards democracy. We model the interactions between the state variables and find that changes in institutional freedoms precedes changes in socio-cultural values. We show applications of our models in studying development traps. This thesis comprises the comprehensive summary and seven papers. Papers I and II describe two similar but complementary methodologies to build non-linear dynamic models from panel datasets. Papers III and IV deal with the demographic transition and policy applications. Papers V and VI describe the transition to democracy and applications. Paper VII describes an application to sustainable development.

The modelling of biological communities is important to further the understanding of species coexistence and the mechanisms involved in maintaining biodiversity. This involves considering not only interactions between individual biological organisms, but also the incorporation of covariate information, if available, in the modelling process. This thesis explores the use of point processes to model interactions in bivariate point patterns within a Bayesian framework, and, where applicable, in conjunction with covariate data. Specifically, we distinguish between symmetric and asymmetric species interactions and model these using appropriate point processes. In this thesis we consider both pairwise and area interaction point processes to allow for inhibitory interactions and both inhibitory and attractive interactions. It is envisaged that the analyses and innovations presented in this thesis will contribute to the parsimonious modelling of biological communities.

published_or_final_version / Statistics / Doctoral / Doctor of Philosophy

published_or_final_version / Statistics / Doctoral / Doctor of Philosophy

published_or_final_version / abstract / Electrical and Electronic Engineering / Doctoral / Doctor of Philosophy

published_or_final_version / Electrical and Electronic Engineering / Master / Master of Philosophy

published_or_final_version / Business / Master / Master of Philosophy

Feature selection is demanded in many modern scientific research problems that use high-dimensional data. A typical example is to find the genes that are most related to a certain disease (e.g., cancer) from high-dimensional gene expression profiles. There are tremendous difficulties in eliminating a large number of useless or redundant features. The expression levels of genes have structure; for example, a group of co-regulated genes that have similar biological functions tend to have similar mRNA expression levels. Many statistical methods have been proposed to take the grouping structure into consideration in feature selection and regression, including Group LASSO, Supervised Group LASSO, and regression on group representatives. In this thesis, we propose to use a sophisticated Markov chain Monte Carlo method (Hamiltonian Monte Carlo with restricted Gibbs sampling) to fit T-probit regression with heavy-tailed priors to make selection in the features with grouping structure. We will refer to this method as fully Bayesian T-probit. The main feature of fully Bayesian T-probit is that it can make feature selection within groups automatically without a pre-specification of the grouping structure and more efficiently discard noise features than LASSO (Least Absolute Shrinkage and Selection Operator). Therefore, the feature subsets selected by fully Bayesian T-probit are significantly more sparse than subsets selected by many other methods in the literature. Such succinct feature subsets are much easier to interpret or understand based on existing biological knowledge and further experimental investigations. In this thesis, we use simulated and real datasets to demonstrate that the predictive performances of the more sparse feature subsets selected by fully Bayesian T-probit are comparable with the much larger feature subsets selected by plain LASSO, Group LASSO, Supervised Group LASSO, random forest, penalized logistic regression and t-test. In addition, we demonstrate that the succinct feature subsets selected by fully Bayesian T-probit have significantly better predictive power than the feature subsets of the same size taken from the top features selected by the aforementioned methods.

For many sequence prediction tasks in Natural Language Processing, modeling dependencies between individual predictions can be used to improve prediction accuracy of the sequence as a whole. Supertagging, involves assigning lexical entries to words based on lexicalized grammatical theory such as Combinatory Categorial Grammar (CCG). Previous work has used Bayesian HMMs to learn taggers for both POS tagging and supertagging separately. Modeling them jointly has the potential to produce more robust and accurate supertaggers trained with less supervision and thereby potentially help in the creation of useful models for new languages and domains. Factorial Hidden Markov Models (FHMM) support joint inference for multiple sequence prediction tasks. Here, I use them to jointly predict part-of-speech tag and supertag sequences with varying levels of supervision. I show that supervised training of FHMM models improves performance compared to standard HMMs, especially when labeled training material is scarce. Secondly, FHMMs trained from tag dictionaries rather than labeled examples also perform better than a standard HMM. Finally, I show that an FHMM and a maximum entropy Markov model can complement each other in a single step co-training setup that improves the performance of both models when there is limited labeled training material available. / text