Sin Ka Yu = 抽取臉孔特徵及辨認臉孔的統計學方法 / 冼家裕. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2004. / Includes bibliographical references (leaves 86-90). / Text in English; abstracts in English and Chinese. / Sin Ka Yu = Chou qu lian kong te zheng ji bian ren lian kong de tong ji xue fang fa / Xian Jiayu. / Chapter Chapter 1. --- Introduction --- p.1 / Chapter 1.1. --- Motivation --- p.1 / Chapter 1.2. --- Objectives --- p.4 / Chapter 1.3. --- Organization of the thesis --- p.4 / Chapter Chapter 2. --- Facial Feature Extraction --- p.6 / Chapter 2.1. --- Introduction --- p.6 / Chapter 2.2. --- Reviews of Statistical Approach --- p.8 / Chapter 2.2.1. --- Eigenfaces --- p.8 / Chapter 2.2.1.1. --- Eigenfeatures Error! Bookmark not defined / Chapter 2.2.3. --- Singular Value Decomposition --- p.14 / Chapter 2.2.4. --- Summary --- p.15 / Chapter 2.3. --- Review of fiducial point localization methods --- p.16 / Chapter 2.3.1. --- Symmetry based Approach --- p.16 / Chapter 2.3.2. --- Color Based Approaches --- p.17 / Chapter 2.3.3. --- Integral Projection --- p.17 / Chapter 2.3.4. --- Deformable Template --- p.20 / Chapter 2.4. --- Corner-based Fiducial Point Localization --- p.22 / Chapter 2.4.1. --- Facial Region Extraction --- p.22 / Chapter 2.4.2. --- Corner Detection --- p.25 / Chapter 2.4.3. --- Corner Selection --- p.27 / Chapter 2.4.3.1. --- Mouth Corner Pairs Detection --- p.27 / Chapter 2.4.3.2. --- Iris Detection --- p.27 / Chapter 2.5. --- Experimental Results --- p.30 / Chapter 2.6. --- Conclusions --- p.30 / Chapter 2.7. --- Notes on Publications --- p.30 / Chapter Chapter 3. --- Fiducial Point Extraction with Shape Constraint --- p.32 / Chapter 3.1. --- Introduction --- p.32 / Chapter 3.2. --- Statistical Theory of Shape --- p.33 / Chapter 3.2.1. --- Shape Space --- p.33 / Chapter 3.2.2. --- Shape Distribution --- p.34 / Chapter 3.3. --- Shape Guided Fiducial Point Localization --- p.38 / Chapter 3.3.1. --- Shape Constraints --- p.38 / Chapter 3.3.2. --- Intelligent Search --- p.40 / Chapter 3.4. --- Experimental Results --- p.40 / Chapter 3.5. --- Conclusions --- p.42 / Chapter 3.6. --- Notes on Publications --- p.42 / Chapter Chapter 4. --- Statistical Pattern Recognition --- p.43 / Chapter 4.1. --- Introduction --- p.43 / Chapter 4.2. --- Bayes Decision Rule --- p.44 / Chapter 4.3. --- Gaussian Maximum Probability Classifier --- p.46 / Chapter 4.4. --- Maximum Likelihood Estimation of Mean and Covariance Matrix --- p.48 / Chapter 4.5. --- Small Sample Size Problem --- p.50 / Chapter 4.5.1. --- Dispersed Eigenvalues --- p.50 / Chapter 4.5.2. --- Distorted Classification Rule --- p.55 / Chapter 4.6. --- Review of Methods Handling the Small Sample Size Problem --- p.57 / Chapter 4.6.1. --- Linear Discriminant Classifier --- p.57 / Chapter 4.6.2. --- Regularized Discriminant Analysis --- p.59 / Chapter 4.6.3. --- Leave-one-out Likelihood Method --- p.63 / Chapter 4.6.4. --- Bayesian Leave-one-out Likelihood method --- p.65 / Chapter 4.7. --- Proposed Method --- p.68 / Chapter 4.7.1. --- A New Covariance Estimator --- p.70 / Chapter 4.7.2. --- Model Selection --- p.75 / Chapter 4.7.3. --- The Mixture Parameter --- p.76 / Chapter 4.8. --- Experimental results --- p.77 / Chapter 4.8.1. --- Implementation --- p.77 / Chapter 4.8.2. --- Results --- p.79 / Chapter 4.9. --- Conclusion --- p.81 / Chapter 4.10. --- Notes on Publications --- p.82 / Chapter Chapter 5. --- Conclusions and Future works --- p.83 / Chapter 5.1. --- Conclusions and Contributions --- p.83 / Chapter 5.2. --- Future Works --- p.84

Wong Hing-Kwok. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2003. / Includes bibliographical references (leaves 34-35). / Abstracts in English and Chinese. / Abstract --- p.i / Acknowledgement --- p.iii / Chapter 1 --- Introduction --- p.1 / Chapter 2 --- Radial Basis Function (RBF) of Neural Network --- p.5 / Chapter 2.1 --- Neural Network --- p.6 / Chapter 2.2 --- Radial Basis Function (RBF) Network --- p.8 / Chapter 2.3 --- Model Specification --- p.10 / Chapter 2.4 --- Estimation --- p.12 / Chapter 3 --- RBF in Performance Attribution --- p.17 / Chapter 3.1 --- Background of Data Set --- p.18 / Chapter 3.2 --- Portfolio Construction --- p.20 / Chapter 3.3 --- Portfolio Rebalance --- p.22 / Chapter 3.4 --- Result --- p.23 / Chapter 4 --- Comparison --- p.26 / Chapter 4.1 --- Standard Linear Model --- p.27 / Chapter 4.2 --- Fixed Additive Model --- p.28 / Chapter 4.3 --- Refined Additive Model --- p.29 / Chapter 4.4 --- Result --- p.30 / Chapter 5 --- Conclusion --- p.32 / Bibliography --- p.34

Computational neuroscience seeks to discover the underlying mechanisms by which neural activity is generated. With the recent advancement in neural data acquisition methods, the bottleneck of this pursuit is the analysis of ever-growing volume of neural data acquired in numerous labs from various experiments. These analyses can be broadly divided into two categories. First, extraction of high quality neuronal signals from noisy large scale recordings. Second, inference for statistical models aimed at explaining the neuronal signals and underlying processes that give rise to them. Conventionally, majority of the methodologies employed for this effort are based on statistics and signal processing. However, in recent years recruiting Artificial Neural Networks (ANN) for neural data analysis is gaining traction. This is due to their immense success in computer vision and natural language processing, and the stellar track record of ANN architectures generalizing to a wide variety of problems. In this work we investigate and improve upon statistical and ANN machine learning methods applied to multi-electrode array recordings and inference for dynamical systems that play critical roles in computational neuroscience. In the first and second part of this thesis, we focus on spike sorting problem. The analysis of large-scale multi-neuronal spike train data is crucial for current and future of neuroscience research. However, this type of data is not available directly from recordings and require further processing to be converted into spike trains. Dense multi-electrode arrays (MEA) are standard methods for collecting such recordings. The processing needed to extract spike trains from these raw electrical signals is carried out by ``spike sorting'' algorithms. We introduce a robust and scalable MEA spike sorting pipeline YASS (Yet Another Spike Sorter) to address many challenges that are inherent to this task. We primarily pay attention to MEA data collected from the primate retina for important reasons such as the unique challenges and available side information that ultimately assist us in scoring different spike sorting pipelines. We also introduce a Neural Network architecture and an accompanying training scheme specifically devised to address the challenging task of deconvolution in MEA recordings. In the last part, we shift our attention to inference for non-linear dynamics. Dynamical systems are the governing force behind many real world phenomena and temporally correlated data. Recently, a number of neural network architectures have been proposed to address inference for nonlinear dynamical systems. We introduce two different methods based on normalizing flows for posterior inference in latent non-linear dynamical systems. We also present gradient-based amortized posterior inference approaches using the auto-encoding variational Bayes framework that can be applied to a wide range of generative models with nonlinear dynamics. We call our method 𝘍𝘪𝘭𝘵𝘦𝘳𝘪𝘯𝘨 𝘕𝘰𝘳𝘮𝘢𝘭𝘪𝘻𝘪𝘯𝘨 𝘍𝘭𝘰𝘸𝘴 (FNF). FNF performs favorably against state-of-the-art inference methods in terms of accuracy of predictions and quality of uncovered codes and dynamics on synthetic data.

No description available.

The work presented in this thesis provides new perspectives and approaches for problems that arise in the analysis of neural data. Particular emphasis is placed on parameter fitting and automated analysis problems that would arise naturally in closed-loop experiments. Part one focuses on two brain-computer interface problems. First, we provide a framework for understanding co-adaptation, the setting in which decoder updating and user learning occur simultaneously. We also provide a new perspective on intention-based parameter fitting and tools to extend this approach to higher dimensional decoders. Part two focuses on event inference, which refers to the decomposition of observed timeseries data into interpretable events. We present application of event inference methods on voltage-clamp recordings as well as calcium imaging, and describe extensions to allow for combining data across modalities or trials.

The emergence of new Internet applications and technologies have resulted in an increased complexity as well as a need for lower latency, higher bandwidth, and increased reliability. This ultimately results in an increased complexity of network operation and management. Manual management is not sufficient to meet these new requirements. There is a need for data driven techniques to advance from manual management to autonomous management of network systems. One such technique, Machine Learning (ML), can use data to create models from hidden patterns in the data and make autonomous modifications. This approach has shown significant improvements in other domains (e.g., image recognition and natural language processing). The use of ML, along with advances in programmable control of Software- Defined Networks (SDNs), will alleviate manual network intervention and ultimately aid in autonomous network operations. However, realizing a data driven system that can not only understand what is happening in the network but also operate autonomously requires advances in the networking domain, as well as in ML algorithms. In this thesis, we focus on developing ML-based network architectures and data driven net- working algorithms whose objective is to improve the performance and management of future networks and network applications. We focus on problems spanning across the network protocol stack from the application layer to the physical layer. We design algorithms and architectures that are motivated by measurements and observations in real world or experimental testbeds. In Part I we focus on the challenge of monitoring and estimating user video quality of experience (QoE) of encrypted video traffic for network operators. We develop a system for REal-time QUality of experience metric detection for Encrypted Traffic, Requet. Requet uses a detection algorithm to identify video and audio chunks from the IP headers of encrypted traffic. Features extracted from the chunk statistics are used as input to a random forest ML model to predict QoE metrics. We evaluate Requet on a YouTube dataset we collected, consisting of diverse video assets delivered over various WiFi and LTE network conditions. We then extend Requet, and present a study on YouTube TV live streaming traffic behavior over WiFi and cellular networks covering a 9-month period. We observed pipelined chunk requests, a reduced buffer capacity, and a more stable chunk duration across various video resolutions compared to prior studies of on-demand streaming services. We develop a YouTube TV analysis tool using chunks statistics detected from the extracted data as input to a ML model to infer user QoE metrics. In Part II we consider allocating end-to-end resources in cellular networks. Future cellular networks will utilize SDN and Network Function Virtualization (NFV) to offer increased flexibility for network infrastructure operators to utilize network resources. Combining these technologies with real-time network load prediction will enable efficient use of network resources. Specifically, we leverage a type of recurrent neural network, Long Short-Term Memory (LSTM) neural networks, for (i) service specific traffic load prediction for network slicing, and (ii) Baseband Unit (BBU) pool traffic load prediction in a 5G cloud Radio Access Network (RAN). We show that leveraging a system with better accuracy to predict service requirements results in a reduction of operation costs. We focus on addressing the optical physical layer in Part III. Greater network flexibility through SDN and the growth of high bandwidth services are motivating faster service provisioning and capacity management in the optical layer. These functionalities require increased capacity along with rapid reconfiguration of network resources. Recent advances in optical hardware can enable a dramatic reduction in wavelength provisioning times in optical circuit switched networks. To support such operations, it is imperative to reconfigure the network without causing a drop in service quality to existing users. Therefore, we present a ML system that uses feedforward neural networks to predict the dynamic response of an optically circuit-switched 90-channel multi-hop Reconfigurable Optical Add-Drop Multiplexer (ROADM) network. We show that the trained deep neural network can recommend wavelength assignments for wavelength switching with minimal power excursions. We extend the performance of the ML system by implementing and testing a Hybrid Machine Learning (HML) model, which combines an analytical model with a neural network machine learning model to achieve higher prediction accuracy. In Part IV, we use a data-driven approach to address the challenge of wireless content delivery in crowded areas. We present the Adaptive Multicast Services (AMuSe) system, whose objective is to enable scalable and adaptive WiFi multicast. Specifically, we develop an algorithm for dynamic selection of a subset of the multicast receivers as feedback nodes. Further, we describe the Multicast Dynamic Rate Adaptation (MuDRA) algorithm that utilizes AMuSe’s feedback to optimally tune the physical layer multicast rate. Our experimental evaluation of MuDRA on the ORBIT testbed shows that MuDRA outperforms other schemes and supports high throughput multicast flows to hundreds of nodes while meeting quality requirements. We leverage the lessons learned from AMuSe for WiFi and use order statistics to address the performance issues with LTE evolved Multimedia Broadcast/Multicast Service (eMBMS). We present the Dynamic Monitoring (DyMo) system which provides low-overhead and real-time feedback about eMBMS performance to be used for network optimization. We focus on the Quality of Service (QoS) Evaluation module and develop a Two-step estimation algorithm which can efficiently identify the SNR Threshold as a one time estimation. DyMo significantly outperforms alternative schemes based on the Order-Statistics estimation method which relies on random or periodic sampling.

Stochasticity arising from data and training can cause statistical errors in prediction and optimization models and lead to inferior decision-making. Understanding the risk associated with the models and converting predictions into better decisions have become increasingly prominent. This thesis studies the interaction of two fundamental topics, data-driven decision-making and machine-learning-based uncertainty reduction, where it develops statistically principled methodologies and provides theoretical insights. Chapter 2 studies data-driven stochastic optimization where model parameters of the underlying distribution need to be estimated from data in addition to the optimization task. Several mainstream approaches have been developed to solve data-driven stochastic optimization, but direct statistical comparisons among different approaches have not been well investigated in the literature. We develop a new regret-based framework based on stochastic dominance to rigorously study and compare their statistical performance. Chapter 3 studies uncertainty quantification and reduction techniques for neural network models. Uncertainties of neural networks arise not only from data, but also from the training procedure that often injects substantial noises and biases. These hinder the attainment of statistical guarantees and, moreover, impose computational challenges due to the need for repeated network retraining. Building upon the recent neural tangent kernel theory, we create statistically guaranteed schemes to principally characterize and remove the uncertainty of over-parameterized neural networks with very low computation effort. Chapter 4 studies reducing uncertainty in stochastic simulation where standard Monte Carlo computation is widely known to exhibit a canonical square-root convergence speed in terms of sample size. Two recent techniques derived from an integration of reproducing kernels and Stein's identity have been proposed to reduce the error in Monte Carlo computation to supercanonical convergence. We present a more general framework to encompass both techniques that is especially beneficial when the sample generator is biased and noise-corrupted. We show that our general estimator, the doubly robust Stein-kernelized estimator, outperforms both existing methods in terms of mean squared error rates across different scenarios. Chapter 5 studies bandit problems, which are important sequential decision-making problems that aim to find optimal adaptive strategies to maximize cumulative reward. Bayesian bandit algorithms with approximate Bayesian inference have been widely used to solve bandit problems in practice, but their theoretical justification is less investigated partially due to the additional Bayesian inference errors. We propose a general theoretical framework to analyze Bayesian bandits in the presence of approximate inference and establish the first regret bound for Bayesian bandit algorithms with bounded approximate inference errors.

In this thesis the two disciplines of Statistics and Artificial Neural Networks are combined into an integrated study of a data set of a weather modification Experiment. An extensive literature study on artificial neural network methodology has revealed the strongly interdisciplinary nature of the research and the applications in this field. An artificial neural networks are becoming increasingly popular with data analysts, statisticians are becoming more involved in the field. A recursive algoritlun is developed to optimize the number of hidden nodes in a feedforward artificial neural network to demonstrate how existing statistical techniques such as nonlinear regression and the likelihood-ratio test can be applied in innovative ways to develop and refine neural network methodology. This pruning algorithm is an original contribution to the field of artificial neural network methodology that simplifies the process of architecture selection, thereby reducing the number of training sessions that is needed to find a model that fits the data adequately. [n addition, a statistical model to classify weather modification data is developed using both a feedforward multilayer perceptron artificial neural network and a discriminant analysis. The two models are compared and the effectiveness of applying an artificial neural network model to a relatively small data set assessed. The formulation of the problem, the approach that has been followed to solve it and the novel modelling application all combine to make an original contribution to the interdisciplinary fields of Statistics and Artificial Neural Networks as well as to the discipline of meteorology. / Mathematical Sciences / D. Phil. (Statistics)

In this thesis the two disciplines of Statistics and Artificial Neural Networks are combined into an integrated study of a data set of a weather modification Experiment. An extensive literature study on artificial neural network methodology has revealed the strongly interdisciplinary nature of the research and the applications in this field. An artificial neural networks are becoming increasingly popular with data analysts, statisticians are becoming more involved in the field. A recursive algoritlun is developed to optimize the number of hidden nodes in a feedforward artificial neural network to demonstrate how existing statistical techniques such as nonlinear regression and the likelihood-ratio test can be applied in innovative ways to develop and refine neural network methodology. This pruning algorithm is an original contribution to the field of artificial neural network methodology that simplifies the process of architecture selection, thereby reducing the number of training sessions that is needed to find a model that fits the data adequately. [n addition, a statistical model to classify weather modification data is developed using both a feedforward multilayer perceptron artificial neural network and a discriminant analysis. The two models are compared and the effectiveness of applying an artificial neural network model to a relatively small data set assessed. The formulation of the problem, the approach that has been followed to solve it and the novel modelling application all combine to make an original contribution to the interdisciplinary fields of Statistics and Artificial Neural Networks as well as to the discipline of meteorology. / Mathematical Sciences / D. Phil. (Statistics)