Wai-tung Ho. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1992. / Includes bibliographical references (leaves 81-83). / Chapter CHAPTER 1 --- OVERVIEW OF CONSTRAINTED ESTIMATION OF STRUCTURAL EQUATION MODEL --- p.1 / Chapter CHAPTER 2 --- MULTISAMPLE ANALYSIS OF STRUCTURAL EQUATION MODELS WITH STOCHASTIC CONSTRAINTS --- p.4 / Chapter 2.1 --- The Basic Model --- p.4 / Chapter 2.2 --- Bayesian Approach to Nuisance Parameters --- p.5 / Chapter 2.3 --- Estimation and Algorithm --- p.8 / Chapter 2.4 --- Asymptotic Properties of the Bayesian Estimate --- p.11 / Chapter CHAPTER 3 --- MULTISAMPLE ANALYSIS OF STRUCTURAL EQUATION MODELS WITH EXACT AND STOCHASTIC CONSTRAINTS --- p.17 / Chapter 3.1 --- The Basic Model --- p.17 / Chapter 3.2 --- Bayesian Approach to Nuisance Parameters and Estimation Procedures --- p.18 / Chapter 3.3 --- Asymptotic Properties of the Bayesian Estimate --- p.20 / Chapter CHAPTER 4 --- SIMULATION STUDIES AND NUMERICAL EXAMPLE --- p.24 / Chapter 4.1 --- Simulation Study for Identified Models with Stochastic Constraints --- p.24 / Chapter 4.2 --- Simulation Study for Non-identified Models with Stochastic Constraints --- p.29 / Chapter 4.3 --- Numerical Example with Exact and Stochastic Constraints --- p.32 / Chapter CHAPTER 5 --- DISCUSSION AND CONCLUSION --- p.34 / APPENDICES --- p.36 / TABLES --- p.66 / REFERENCES --- p.81

by Chi-van Lam. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1993. / Includes bibliographical references (leaves 71-72). / Chapter Chapter 1 --- Introduction --- p.1 / Chapter Chapter 2 --- The Model and the Bayes risk --- p.7 / Chapter § 2.1 --- The Model / Chapter § 2.2 --- The Bayes risk / Chapter Chapter 3 --- The Algorithm --- p.16 / Chapter § 3.1 --- A finite algorithm / Chapter § 3.2 --- The Number Theoretical Method for Optimization / Chapter § 3.2.1 --- NTMO / Chapter § 3.2.2 --- SNTMO / Chapter Chapter 4 --- Quadratic Loss Function --- p.26 / Chapter §4.1 --- The Bayes risk / Chapter § 4.2 --- An optimal plan / Chapter § 4.3 --- Numerical Examples / Chapter Chapter 5 --- Conclusions and Comments --- p.42 / Chapter § 5.1 --- Comparison between various plans / Chapter § 5.2 --- Sensitivity Analysis / Chapter § 5.3 --- Further Developments / Tables --- p.46 / Appendix A --- p.60 / Appendix B --- p.65 / References --- p.71

This thesis investigates the use of Bayesian Networks (BNs), augmented by the Dynamic Discretization Algorithm, to model a variety of clinical problems. In particular, the thesis demonstrates four novel applications of BN and dynamic discretization to clinical problems. Firstly, it demonstrates the flexibility of the Dynamic Discretization Algorithm in modeling existing medical knowledge using appropriate statistical distributions. Many practical applications of BNs use the relative frequency approach while translating existing medical knowledge to a prior distribution in a BN model. This approach does not capture the full uncertainty surrounding the prior knowledge. Secondly, it demonstrates a novel use of the multinomial BN formulation in learning parameters of categorical variables. The traditional approach requires fixed number of parameters during the learning process but this framework allows an analyst to generate a multinomial BN model based on the number of parameters required. Thirdly, it presents a novel application of the multinomial BN formulation and dynamic discretization to learning causal relations between variables. The idea is to consider competing causal relations between variables as hypotheses and use data to identify the best hypothesis. The result shows that BN models can provide an alternative to the conventional causal learning techniques. The fourth novel application is the use of Hierarchical Bayesian Network (HBN) models, augmented by dynamic discretization technique, to meta-analysis of clinical data. The result shows that BN models can provide an alternative to classical meta analysis techniques. The thesis presents two clinical case studies to demonstrate these novel applications of BN models. The first case study uses data from a multi-disciplinary team at the Royal London hospital to demonstrate the flexibility of the multinomial BN framework in learning parameters of a clinical model. The second case study demonstrates the use of BN and dynamic discretization to solving decision problem. In summary, the combination of the Junction Tree Algorithm and Dynamic Discretization Algorithm provide a unified modeling framework for solving interesting clinical problems.

One of the hardest challenges in building a realistic Bayesian network (BN) model is to construct the node probability tables (NPTs). Even with a fixed predefined model structure and very large amounts of relevant data, machine learning methods do not consistently achieve great accuracy compared to the ground truth when learning the NPT entries (parameters). Hence, it is widely believed that incorporating expert judgment or related domain knowledge can improve the parameter learning accuracy. This is especially true in the sparse data situation. Expert judgments come in many forms. In this thesis we focus on expert judgment that specifies inequality or equality relationships among variables. Related domain knowledge is data that comes from a different but related problem. By exploiting expert judgment and related knowledge, this thesis makes novel contributions to improve the BN parameter learning performance, including: • The multinomial parameter learning model with interior constraints (MPL-C) and exterior constraints (MPL-EC). This model itself is an auxiliary BN, which encodes the multinomial parameter learning process and constraints elicited from the expert judgments. • The BN parameter transfer learning (BNPTL) algorithm. Given some potentially related (source) BNs, this algorithm automatically explores the most relevant source BN and BN fragments, and fuses the selected source and target parameters in a robust way. • A generic BN parameter learning framework. This framework uses both expert judgments and transferred knowledge to improve the learning accuracy. This framework transfers the mined data statistics from the source network as the parameter priors of the target network. Experiments based on the BNs from a well-known repository as well as two realworld case studies using different data sample sizes demonstrate that the proposed new approaches can achieve much greater learning accuracy compared to other state-of-theart methods with relatively sparse data.

Low Kon Fan. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1998. / Includes bibliographical references (leaves 165-168). / Abstract also in Chinese. / Abstract --- p.i / Acknowledgment --- p.iv / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Overview of Information Filtering --- p.1 / Chapter 1.2 --- Contributions --- p.4 / Chapter 1.3 --- Organization of this thesis --- p.6 / Chapter 2 --- Existing Approaches --- p.7 / Chapter 2.1 --- Representational issues --- p.7 / Chapter 2.1.1 --- Document Representation --- p.7 / Chapter 2.1.2 --- Feature Selection --- p.11 / Chapter 2.2 --- Traditional Approaches --- p.15 / Chapter 2.2.1 --- NewsWeeder --- p.15 / Chapter 2.2.2 --- NewT --- p.17 / Chapter 2.2.3 --- SIFT --- p.19 / Chapter 2.2.4 --- InRoute --- p.20 / Chapter 2.2.5 --- Motivation of Our Approach --- p.21 / Chapter 2.3 --- Probabilistic Approaches --- p.23 / Chapter 2.3.1 --- The Naive Bayesian Approach --- p.25 / Chapter 2.3.2 --- The Bayesian Independence Classifier Approach --- p.28 / Chapter 2.4 --- Comparison --- p.31 / Chapter 3 --- Our Bayesian Network Approach --- p.33 / Chapter 3.1 --- Backgrounds of Bayesian Networks --- p.34 / Chapter 3.2 --- Bayesian Network Induction Approach --- p.36 / Chapter 3.3 --- Automatic Construction of Bayesian Networks --- p.38 / Chapter 4 --- Automatic Feature Discretization --- p.50 / Chapter 4.1 --- Predefined Level Discretization --- p.52 / Chapter 4.2 --- Lloyd's algorithm . . > --- p.53 / Chapter 4.3 --- Class Dependence Discretization --- p.55 / Chapter 5 --- Experiments and Results --- p.59 / Chapter 5.1 --- Document Collections --- p.60 / Chapter 5.2 --- Batch Filtering Experiments --- p.63 / Chapter 5.3 --- Batch Filtering Results --- p.65 / Chapter 5.4 --- Incremental Session Filtering Experiments --- p.87 / Chapter 5.5 --- Incremental Session Filtering Results --- p.88 / Chapter 6 --- Conclusions and Future Work --- p.105 / Appendix A --- p.107 / Appendix B --- p.116 / Appendix C --- p.126 / Appendix D --- p.131 / Appendix E --- p.145

Imagine a world where computational simulations can be inverted as easily as running them forwards, where data can be used to refine models automatically, and where the only expertise one needs to carry out powerful statistical analysis is a basic proficiency in scientific coding. Creating such a world is the ambitious long-term aim of probabilistic programming. The bottleneck for improving the probabilistic models, or simulators, used throughout the quantitative sciences, is often not an ability to devise better models conceptually, but a lack of expertise, time, or resources to realize such innovations. Probabilistic programming systems (PPSs) help alleviate this bottleneck by providing an expressive and accessible modeling framework, then automating the required computation to draw inferences from the model, for example finding the model parameters likely to give rise to a certain output. By decoupling model specification and inference, PPSs streamline the process of developing and drawing inferences from new models, while opening up powerful statistical methods to non-experts. Many systems further provide the flexibility to write new and exciting models which would be hard, or even impossible, to convey using conventional statistical frameworks. The central goal of this thesis is to improve and extend PPSs. In particular, we will make advancements to the underlying inference engines and increase the range of problems which can be tackled. For example, we will extend PPSs to a mixed inference-optimization framework, thereby providing automation of tasks such as model learning and engineering design. Meanwhile, we make inroads into constructing systems for automating adaptive sequential design problems, providing potential applications across the sciences. Furthermore, the contributions of the work reach far beyond probabilistic programming, as achieving our goal will require us to make advancements in a number of related fields such as particle Markov chain Monte Carlo methods, Bayesian optimization, and Monte Carlo fundamentals.

Gaussian processes can be used as priors on functions. The need for a flexible, principled, probabilistic model of functional relations is common in practice. Consequently, such an approach is demonstrably useful in a large variety of applications. Two challenges of Gaussian process modelling are often encountered. These are dealing with the adverse scaling with the number of data points and the lack of closed form posteriors when the likelihood is non-Gaussian. In this thesis, we study variational inference as a framework for meeting these challenges. An introductory chapter motivates the use of stochastic processes as priors, with a particular focus on Gaussian process modelling. A section on variational inference reviews the general definition of Kullback-Leibler divergence. The concept of prior conditional matching that is used throughout the thesis is contrasted to classical approaches to obtaining tractable variational approximating families. Various theoretical issues arising from the application of variational inference to the infinite dimensional Gaussian process setting are settled decisively. From this theory we are able to give a new argument for existing approaches to variational regression that settles debate about their applicability. This view on these methods justifies the principled extensions found in the rest of the work. The case of scalable Gaussian process classification is studied, both for its own merits and as a case study for non-Gaussian likelihoods in general. Using the resulting algorithms we find credible results on datasets of a scale and complexity that was not possible before our work. An extension to include Bayesian priors on model hyperparameters is studied alongside a new inference method that combines the benefits of variational sparsity and MCMC methods. The utility of such an approach is shown on a variety of example modelling tasks. We describe GPflow, a new Gaussian process software library that uses TensorFlow. Implementations of the variational algorithms discussed in the rest of the thesis are included as part of the software. We discuss the benefits of GPflow when compared to other similar software. Increased computational speed is demonstrated in relevant, timed, experimental comparisons.

Au Siu Chun Brian. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2006. / Includes bibliographical references (leaves 54-57). / Abstracts in English and Chinese. / Chapter 1 --- Introduction --- p.1 / Chapter 2 --- Related sampling methods --- p.4 / Chapter 2.1 --- Introduction --- p.4 / Chapter 2.2 --- Gibbs sampler --- p.4 / Chapter 2.3 --- Importance sampling --- p.5 / Chapter 2.4 --- Sampling-importance resampling (SIR) --- p.7 / Chapter 2.5 --- Inverse Bayes formulae sampling (IBF sampling) --- p.10 / Chapter 3 --- Resampling weights in the SIR algorithm --- p.13 / Chapter 3.1 --- Resampling weights --- p.13 / Chapter 3.2 --- Problem in IBF sampling --- p.18 / Chapter 3.3 --- Adaptive finite mixture of distributions --- p.18 / Chapter 3.4 --- Allowing general distribution of 9 --- p.21 / Chapter 3.5 --- Examples and graphical comparison --- p.24 / Chapter 4 --- Resampling weight in Gibbs sampling --- p.32 / Chapter 4.1 --- Introduction --- p.32 / Chapter 4.2 --- Use Gibbs sampler to obtain ISF --- p.33 / Chapter 4.3 --- How many iterations? --- p.36 / Chapter 4.4 --- Applications --- p.41 / Chapter 4.4.1 --- The genetic linkage model --- p.41 / Chapter 4.4.2 --- Example --- p.43 / Chapter 4.4.3 --- The probit binary regression model --- p.44 / Chapter 5 --- Conclusion and discussion --- p.49 / Appendix A: Exact bias of the SIR --- p.52 / References --- p.54

在貝葉斯推理框架下，貝葉斯方法可以通過數據推斷複雜概率模型中的參數和結構。它被廣泛應用於多个領域。對於生物信息學問題，貝葉斯方法同樣也是一個理想的方法。本文通過介紹新的貝葉斯模型和計算方法討論並解決了兩個與模型選擇相關的生物信息學問題。 / 第一個問題是關於在DNA 序列中的模式識別的相關研究。串聯重複序列片段在DNA 序列中經常出現。它對於基因組進化和人類疾病的研究非常重要。在這一部分，本文主要討論不確定數目的同一模式的串聯重複序列彌散分佈在同一個序列中的情況。我們首先對串聯重複序列片段構建概率模型。然後利用馬爾可夫鏈蒙特卡羅算法探索後驗分佈進而推斷出串聯重複序列的重複片段的模式矩陣和位置。此外，利用RJMCMC 算法解決由不確定數目的重複片段引起的模型選擇問題。 / 另一個問題是對於生物分子的構象轉換的分析。一組生物分子的構象可被分成幾個不同的亞穩定狀態。由於生物分子的功能和構象之間的固有聯繫，構象轉變在不同的生物分子的生物過程中都扮演者非常重要的角色。一般我們從分子動力學模擬中可以得到構象轉換的數據。基於從分子動力學模擬中得到的微觀狀態水準上的構象轉換資訊，我們利用貝葉斯方法研究從微觀狀態到可變數目的亞穩定狀態的聚合問題。 / 本文通過對以上兩個問題討論闡釋貝葉斯方法在生物信息學研究的多個方面具備優勢。這包括闡述生物問題的多變性，處理噪聲和失數據，以及解決模型選擇問題。 / Bayesian approach is a powerful framework for inferring the parameters and structures of complicated probabilistic models from data. It is widely applied in many areas and also ideal for Bioinformatics problems due to their usually high complexity. In this thesis, new Bayesian models and computing methods are introduced to solve two Bioinformatics problems which are both related to model selection. / The first problem is about the repeat pattern recognition. Tandem repeats occur frequently in DNA sequences. They are important for studying genome evolution and human disease. This thesis focuses on the case that an unknown number of tandem repeat segments of the same pattern are dispersively distributed in a sequence. A probabilistic generative model is introduced for the tandem repeats. Markov chain Monte Carlo algorithms are used to explore the posterior distribution as an effort to infer both the specific pattern of the tandem repeats and the location of repeat segments. Furthermore, reversible jump Markov chain Monte Carlo algorithms are used to address the transdimensional model selection problem raised by the variable number of repeat segments. / The second part of this thesis is engaged in the conformational transitions of biomolecules. Because the function of a biological biomolecule is inherently related to its variable conformations which can be grouped into a set of metastable or long-live states, conformational transitions are important in biological processes. The 3D structure changes are generally simulated from the molecular dynamics computer simulation. Based on the conformational transitions on microstate level from molecular dynamics simulation, a Bayesian approach is developed to cluster the microstates into an uncertainty number of metastable that induces the model selection problem. / With these two problems, this thesis shows that the Bayesian approach for bioinformatics problems has its advantages in terms of taking account of the inherent uncertainty in biological data, handling noisy or missing data, and dealing with the model selection problem. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Detailed summary in vernacular field only. / Liang, Tong. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2013. / Includes bibliographical references (leaves 120-130). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstracts also in Chinese. / Abstract --- p.i / Acknowledgement --- p.iv / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Motivation --- p.1 / Chapter 1.2 --- Statistical Background --- p.2 / Chapter 1.3 --- Tandem Repeats --- p.4 / Chapter 1.4 --- Conformational Space --- p.5 / Chapter 1.5 --- Outlines --- p.7 / Chapter 2 --- Preliminaries --- p.9 / Chapter 2.1 --- Bayesian Inference --- p.9 / Chapter 2.2 --- Markov chain Monte Carlo --- p.10 / Chapter 2.2.1 --- Gibbs sampling --- p.11 / Chapter 2.2.2 --- Metropolis - Hastings algorithm --- p.12 / Chapter 2.2.3 --- Reversible Jump MCMC --- p.12 / Chapter 3 --- Detection of Dispersed Short Tandem Repeats Using Reversible Jump MCMC --- p.14 / Chapter 3.1 --- Background --- p.14 / Chapter 3.2 --- Generative Model --- p.17 / Chapter 3.3 --- Statistical inference --- p.18 / Chapter 3.3.1 --- Likelihood --- p.19 / Chapter 3.3.2 --- Prior Distributions --- p.19 / Chapter 3.3.3 --- Sampling from Posterior Distribution via RJMCMC --- p.20 / Chapter 3.3.4 --- Extra MCMC moves for better mixing --- p.26 / Chapter 3.3.5 --- The complete algorithm --- p.29 / Chapter 3.4 --- Experiments --- p.29 / Chapter 3.4.1 --- Evaluation and comparison of the two RJMCMC versions using synthetic data --- p.30 / Chapter 3.4.2 --- Comparison with existing methods using synthetic data --- p.33 / Chapter 3.4.3 --- Sensitivity to Priors --- p.43 / Chapter 3.4.4 --- Real data experiment --- p.45 / Chapter 3.5 --- Discussion --- p.50 / Chapter 4 --- A Probabilistic Clustering Algorithm for Conformational Changes of Biomolecules --- p.53 / Chapter 4.1 --- Introduction --- p.53 / Chapter 4.1.1 --- Molecular dynamic simulation --- p.54 / Chapter 4.1.2 --- Hierarchical Conformational Space --- p.55 / Chapter 4.1.3 --- Clustering Algorithms --- p.56 / Chapter 4.2 --- Generative Model --- p.58 / Chapter 4.2.1 --- Model 1: Vanilla Model --- p.59 / Chapter 4.2.2 --- Model 2: Zero-Inflated Model --- p.60 / Chapter 4.2.3 --- Model 3: Constrained Model --- p.61 / Chapter 4.2.4 --- Model 4: Constrained and Zero-Inflated Model --- p.61 / Chapter 4.3 --- Statistical Inference for Vanilla Model --- p.62 / Chapter 4.3.1 --- Priors --- p.62 / Chapter 4.3.2 --- Posterior distribution --- p.63 / Chapter 4.3.3 --- Collapsed Gibbs for Vanilla Model with a Fixed Number of Clusters --- p.63 / Chapter 4.3.4 --- Inference on the Number of Clusters --- p.65 / Chapter 4.3.5 --- Synthetic Data Study --- p.68 / Chapter 4.4 --- Statistical Inference for Zero-Inflated Model --- p.76 / Chapter 4.4.1 --- Method 1 --- p.78 / Chapter 4.4.2 --- Method 2 --- p.81 / Chapter 4.4.3 --- Synthetic Data Study --- p.84 / Chapter 4.5 --- Statistical Inference for Constrained Model --- p.85 / Chapter 4.5.1 --- Priors --- p.85 / Chapter 4.5.2 --- Posterior Distribution --- p.86 / Chapter 4.5.3 --- Collapsed Posterior Distribution --- p.86 / Chapter 4.5.4 --- Updating for Cluster Labels K --- p.89 / Chapter 4.5.5 --- Updating for Constrained Λ from Truncated Distribution --- p.89 / Chapter 4.5.6 --- Updating the Number of Clusters --- p.91 / Chapter 4.5.7 --- Uniform Background Parameters on Λ --- p.92 / Chapter 4.6 --- Real Data Experiments --- p.93 / Chapter 4.7 --- Discussion --- p.104 / Chapter 5 --- Conclusion and FutureWork --- p.107 / Chapter A --- Appendix --- p.109 / Chapter A.1 --- Post-processing for indel treatment --- p.109 / Chapter A.2 --- Consistency Score --- p.111 / Chapter A.3 --- A Proof for Collapsed Posterior distribution in Constrained Model in Chapter 4 --- p.111 / Chapter A.4 --- Estimated Transition Matrices for Alanine Dipeptide by Chodera et al. (2006) --- p.117 / Bibliography --- p.120

Air pollution is an important determinant of health. There is convincing, and growing, evidence linking the risk of disease, and premature death, with exposure to various pollutants including fine particulate matter and ozone. Knowledge about the health and environmental risks and their trends is important stimulus for developing environmental and public health policy. In order to perform studies into the risks of environmental hazards on human health study there is a requirement for accurate estimates of exposures that might be experienced by the populations at risk. In this thesis we develop spatio-temporal models within a Bayesian framework to obtain accurate estimates of such exposures. These models are set within a hierarchical framework in a Bayesian setting with different levels describing dependencies over space and time. Considering the complexity of hierarchical models and the large amounts of data that can arise from environmental networks mean that inference using Markov Chain Monte Carlo (MCMC) may be computational challenging in this setting. We use both MCMC and Integrated Nested Laplace Approximations (INLA) to implement spatio-temporal exposure models when dealing with high–dimensional data. We also propose an approach for utilising the results from exposure models in health models which allows them to enhance studies of the health effects of air pollution. Moreover, we investigate the possible effects of preferential sampling, where monitoring sites in environmental networks are preferentially located by the designers in order to assess whether guideline and policies are being adhered to. This means the data arising from such networks may not accurately characterise the spatial-temporal field they intend to monitor and as such will not provide accurate estimates of the exposures that are potentially experienced by populations. This has the potential to introduce bias into estimates of risk associated with exposure to air pollution and subsequent health impact analyses. Throughout the thesis, the methods developed are assessed using simulation studies and applied to real–life case studies assessing the effects of particulate matter on health in Greater London and throughout the UK.