Faculty of Engineering and Built Enviroment School of Civil and Enviromental Engineering 0105772w imasengo@yahoo.com / The problem of missing data, insufficient length of hydrological data series and poor quality is common in developing countries. This problem is much more prevalent in developing countries than it is in developed countries. This situation can severely affect the outcome of the water systems managers’ decisions (e.g. reliability of the design, establishment of operating policies for water supply, etc). Thus, numerous data interpolation (infilling) techniques have evolved in hydrology to deal with the missing data. The current study presents merely a methodology by combining different approaches and coping with missing (limited) hydrological data using the theories of entropy, artificial neural networks (ANN) and expectation-maximization (EM) techniques. This methodology is simply formulated into a model named ENANNEX model. This study does not use any physical characteristics of the catchment areas but deals only with the limited information (e.g. streamflow or rainfall) at the target gauge and its similar nearby base gauge(s). The entropy concept was confirmed to be a versatile tool. This concept was firstly used for quantifying information content of hydrological variables (e.g. rainfall or streamflow). The same concept (through directional information transfer index, i.e. DIT) was used in the selection of base/subject gauge. Finally, the DIT notion was also extended to the evaluation of the hydrological data infilling technique performance (i.e. ANN and EM techniques). The methodology was applied to annual total rainfall; annual mean flow series, annual maximum flows and 6-month flow series (means) of selected catchments in the drainage region D “Orange” of South Africa. These data regimes can be regarded as useful for design-oriented studies, flood studies, water balance studies, etc. The results from the case studies showed that DIT is as good index for data infilling technique selection as other criteria, e.g. statistical and graphical. However, the DIT has the feature of being non-dimensionally informational index. The data interpolation iii techniques viz. ANNs and EM (existing methods applied and not yet applied in hydrology) and their new features have been also presented. This study showed that the standard techniques (e.g. Backpropagation-BP and EM) as well as their respective variants could be selected in the missing hydrological data estimation process. However, the capability for the different data interpolation techniques of maintaining the statistical characteristics (e.g. mean, variance) of the target gauge was not neglected. From this study, the relationship between the accuracy of the estimated series (by applying a data infilling technique) and the gap duration was then investigated through the DIT notion. It was shown that a decay (power or exponential) function could better describe that relationship. In other words, the amount of uncertainty removed from the target station in a station-pair, via a given technique, could be known for a given gap duration. It was noticed that the performance of the different techniques depends on the gap duration at the target gauge, the station-pair involved in the missing data estimation and the type of the data regime. This study showed also that it was possible, through entropy approach, to assess (preliminarily) model performance for simulating runoff data at a site where absolutely no record exist: a case study was conducted at Bedford site (in South Africa). Two simulation models, viz. RAFLER and WRSM2000 models, were then assessed in this respect. Both models were found suitable for simulating flows at Bedford.

In Natural Language Processing (NLP), speech and text are parsed and generated with language models and parser models, and translated with translation models. Each model contains a set of numerical parameters which are found by applying a suitable training algorithm to a set of training data. Many such training algorithms are instances of the Expectation-Maximization (EM) algorithm. In [BSV15], a generic EM algorithm for NLP is described. This work presents a particular speech model, the Hidden Markov model, and its standard training algorithm, the Baum-Welch algorithm. It is then shown that the Baum-Welch algorithm is an instance of the generic EM algorithm introduced by [BSV15], from which follows that all statements about the generic EM algorithm also apply to the Baum-Welch algorithm, especially its correctness and convergence properties.

In this thesis appropriate statistical methods to overcome two types of problems that occur during parameter estimation in chemical engineering systems are studied. The first problem is having too many parameters to estimate from limited available data, assuming that the model structure is correct, while the second problem involves estimating unmeasured disturbances, assuming that enough data are available for parameter estimation. In the first part of this thesis, a model is developed to predict rates of undesirable reactions during the finishing stage of nylon 66 production. This model has too many parameters to estimate (56 unknown parameters) and not having enough data to reliably estimating all of the parameters. Statistical techniques are used to determine that 43 of 56 parameters should be estimated. The proposed model matches the data well. In the second part of this thesis, techniques are proposed for estimating parameters in Stochastic Differential Equations (SDEs). SDEs are fundamental dynamic models that take into account process disturbances and model mismatch. Three new approximate maximum likelihood methods are developed for estimating parameters in SDE models. First, an Approximate Expectation Maximization (AEM) algorithm is developed for estimating model parameters and process disturbance intensities when measurement noise variance is known. Then, a Fully-Laplace Approximation Expectation Maximization (FLAEM) algorithm is proposed for simultaneous estimation of model parameters, process disturbance intensities and measurement noise variances in nonlinear SDEs. Finally, a Laplace Approximation Maximum Likelihood Estimation (LAMLE) algorithm is developed for estimating measurement noise variances along with model parameters and disturbance intensities in nonlinear SDEs. The effectiveness of the proposed algorithms is compared with a maximum-likelihood based method. For the CSTR examples studied, the proposed algorithms provide more accurate estimates for the parameters. Additionally, it is shown that the performance of LAMLE is superior to the performance of FLAEM. SDE models and associated parameter estimates obtained using the proposed techniques will help engineers who implement on-line state estimation and process monitoring schemes. / Thesis (Ph.D, Chemical Engineering) -- Queen's University, 2013-12-23 15:12:35.738

by Long Mei. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2005. / Includes bibliographical references (leaves 73-77). / Abstracts in English and Chinese. / Acknowledgements --- p.iv / Abstract --- p.v / Table of Contents --- p.vii / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Nonlinear Factor Analysis Model --- p.1 / Chapter 1.2 --- Main Objectives --- p.2 / Chapter 1.2.1 --- Investigation of the performance of the ML approach with MCECM algorithm in NFA model --- p.2 / Chapter 1.2.2 --- Investigation of the Robustness of the ML approach with MCECM algorithm --- p.3 / Chapter 1.3 --- Structure of the Thesis --- p.3 / Chapter 2 --- Theoretical Background of the MCECM Algorithm --- p.5 / Chapter 2.1 --- Introduction of the EM algorithm --- p.5 / Chapter 2.2 --- Monte Carlo integration --- p.7 / Chapter 2.3 --- Markov Chains --- p.7 / Chapter 2.4 --- The Metropolis-Hastings algorithm --- p.8 / Chapter 3 --- Maximum Likelihood Estimation of a Nonlinear Factor Analysis Model --- p.10 / Chapter 3.1 --- MCECM Algorithm --- p.10 / Chapter 3.1.1 --- Motivation of Using MCECM algorithm --- p.11 / Chapter 3.1.2 --- Introduction of the Realization of the MCECM algorithm --- p.12 / Chapter 3.1.3 --- Implementation of the E-step via the MH Algorithm --- p.13 / Chapter 3.1.4 --- Maximization Step --- p.15 / Chapter 3.2 --- Monitoring Convergence of MCECM --- p.17 / Chapter 3.2.1 --- Bridge Sampling Method --- p.17 / Chapter 3.2.2 --- Average Batch Mean Method --- p.18 / Chapter 4 --- Simulation Studies --- p.20 / Chapter 4.1 --- The First Simulation Study with the Normal Distribution --- p.20 / Chapter 4.1.1 --- Model Specification --- p.20 / Chapter 4.1.2 --- The Selection of System Parameters --- p.22 / Chapter 4.1.3 --- Monitoring the Convergence --- p.22 / Chapter 4.1.4 --- Simulation Results for the ML Estimates --- p.25 / Chapter 4.2 --- The Second Simulation Study with the Normal Distribution --- p.34 / Chapter 4.2.1 --- Model Specification --- p.34 / Chapter 4.2.2 --- Monitoring the Convergence --- p.35 / Chapter 4.2.3 --- Simulation Results for the ML Estimates --- p.38 / Chapter 4.3 --- The Third Simulation Study on Robustness --- p.47 / Chapter 4.3.1 --- Model Specification --- p.47 / Chapter 4.3.2 --- Monitoring the Convergence --- p.48 / Chapter 4.3.3 --- Simulation Results for the ML Estimates --- p.51 / Chapter 4.4 --- The Fourth Simulation Study on Robustness --- p.59 / Chapter 4.4.1 --- Model Specification --- p.59 / Chapter 4.4.2 --- Monitoring the Convergence --- p.59 / Chapter 4.4.3 --- Simulation Results for the ML Estimates --- p.62 / Chapter 5 --- Conclusion --- p.71 / Bibliography --- p.73

Bin Lu. / "September 2004." / Thesis (Ph.D.)--Chinese University of Hong Kong, 2004. / Includes bibliographical references (p. 118-126) / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Mode of access: World Wide Web. / Abstracts in English and Chinese.

Learning the parts of objects have drawn more attentions in computer science recently, and they have been playing the important role in computer applications such as object recognition, self-driving cars, and image processing, etc… However, the existing research such as traditional non-negative matrix factorization (NMF), principal component analysis (PCA), and vector quantitation (VQ) has not been discovering the ground-truth bases which are basic components representing objects. On this thesis, I am proposed to study on pattern recognition enhancement combined non-negative matrix factorization (NMF) with automatic relevance determination (ARD). The main point of this research is to propose a new technique combining the algorithm Expectation Maximization (EM) with Automatic Relevance Determination (ARD) to discover the ground truth basis of datasets, and then to compare my new proposed technique to the others such as: traditional NMF, sparseness constraint and graph embedding in pattern recognition problems to verify if my method has over performance in accuracy rate than the others. Particularly, the new technique will be tested on variety of datasets from simple to complex one, from synthetic datasets to real ones. To compare the performance, I split these datasets into 10 random partitions as the training and the testing sets called 10-fold cross validation, and then use the technique called Euclidean algorithm to classify them and test their accuracy. As the result, my proposed method has higher accuracy than the others, and it is good to use in pattern recognition problems with missing data.

<p>KTH, Division of Neuronic Engineering, have a finite element model of the head. However, this model does not contain detailed modeling of the brain. This thesis project consists of finding a method to extract brain tissues from T1-weighted MR images of the head. The method should be automatic to be suitable for patient individual modeling.</p><p>A summary of the most common segmentation methods is presented and one of the methods is implemented. The implemented method is based on the assumption that the probability density function (pdf) of an MR image can be described by parametric models. The intensity distribution of each tissue class is modeled as a Gaussian distribution. Thus, the total pdf is a sum of Gaussians. However, the voxel values are also influenced by intensity inhomogeneities, which affect the pdf. The implemented method is based on the expectation-maximization algorithm and it corrects for intensity inhomogeneities. The result from the algorithm is a classification of the voxels. The brain is extracted from the classified voxels using morphological operations.</p>

University of Macau / Faculty of Science and Technology / Department of Electrical and Electronics Engineering

In drug discovery, thousands of compounds are assayed to detect activity against a biological target. The goal of drug discovery is to identify compounds that are active against the target (e.g. inhibit a virus). Statistical learning in drug discovery seeks to build a model that uses descriptors characterizing molecular structure to predict biological activity. However, the characteristics of drug discovery data can make it difficult to model the relationship between molecular descriptors and biological activity. Among these characteristics are the rarity of active compounds, the large volume of compounds tested by high-throughput screening, and the complexity of molecular structure and its relationship to activity. This thesis focuses on the design of statistical learning algorithms/models and their applications to drug discovery. The two main parts of the thesis are: an algorithm-based statistical method and a more formal model-based approach. Both approaches can facilitate and accelerate the process of developing new drugs. A unifying theme is the use of unsupervised methods as components of supervised learning algorithms/models. In the first part of the thesis, we explore a sequential screening approach, Cluster Structure-Activity Relationship Analysis (CSARA). Sequential screening integrates High Throughput Screening with mathematical modeling to sequentially select the best compounds. CSARA is a cluster-based and algorithm driven method. To gain further insight into this method, we use three carefully designed experiments to compare predictive accuracy with Recursive Partitioning, a popular structureactivity relationship analysis method. The experiments show that CSARA outperforms Recursive Partitioning. Comparisons include problems with many descriptor sets and situations in which many descriptors are not important for activity. In the second part of the thesis, we propose and develop constrained mixture discriminant analysis (CMDA), a model-based method. The main idea of CMDA is to model the distribution of the observations given the class label (e.g. active or inactive class) as a constrained mixture distribution, and then use Bayes’ rule to predict the probability of being active for each observation in the testing set. Constraints are used to deal with the otherwise explosive growth of the number of parameters with increasing dimensionality. CMDA is designed to solve several challenges in modeling drug data sets, such as multiple mechanisms, the rare target problem (i.e. imbalanced classes), and the identification of relevant subspaces of descriptors (i.e. variable selection). We focus on the CMDA1 model, in which univariate densities form the building blocks of the mixture components. Due to the unboundedness of the CMDA1 log likelihood function, it is easy for the EM algorithm to converge to degenerate solutions. A special Multi-Step EM algorithm is therefore developed and explored via several experimental comparisons. Using the multi-step EM algorithm, the CMDA1 model is compared to model-based clustering discriminant analysis (MclustDA). The CMDA1 model is either superior to or competitive with the MclustDA model, depending on which model generates the data. The CMDA1 model has better performance than the MclustDA model when the data are high-dimensional and unbalanced, an essential feature of the drug discovery problem! An alternate approach to the problem of degeneracy is penalized estimation. By introducing a group of simple penalty functions, we consider penalized maximum likelihood estimation of the CMDA1 and CMDA2 models. This strategy improves the convergence of the conventional EM algorithm, and helps avoid degenerate solutions. Extending techniques from Chen et al. (2007), we prove that the PMLE’s of the two-dimensional CMDA1 model can be asymptotically consistent.

KTH, Division of Neuronic Engineering, have a finite element model of the head. However, this model does not contain detailed modeling of the brain. This thesis project consists of finding a method to extract brain tissues from T1-weighted MR images of the head. The method should be automatic to be suitable for patient individual modeling. A summary of the most common segmentation methods is presented and one of the methods is implemented. The implemented method is based on the assumption that the probability density function (pdf) of an MR image can be described by parametric models. The intensity distribution of each tissue class is modeled as a Gaussian distribution. Thus, the total pdf is a sum of Gaussians. However, the voxel values are also influenced by intensity inhomogeneities, which affect the pdf. The implemented method is based on the expectation-maximization algorithm and it corrects for intensity inhomogeneities. The result from the algorithm is a classification of the voxels. The brain is extracted from the classified voxels using morphological operations.