by Lee Wai Hung. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1999. / Includes bibliographical references (leaves 118-121). / Abstracts in English and Chinese. / Chapter 1 --- INTRODUCTION --- p.1 / Chapter 1.1 --- Background --- p.1 / Chapter 1.2 --- Structure of Thesis --- p.8 / Chapter 1.3 --- Methodology --- p.8 / Chapter 2 --- BACKGROUND THEORY --- p.10 / Chapter 2.1 --- Radio Wave Propagation Modeling --- p.10 / Chapter 2.1.1 --- Basic Propagation Phenomena --- p.10 / Chapter 2.1.1.1 --- Propagation in Free Space --- p.10 / Chapter 2.1.1.2 --- Reflection and Transmission --- p.11 / Chapter 2.1.2 --- Practical Propagation Models --- p.12 / Chapter 2.1.2.1 --- Longley-Rice Model --- p.13 / Chapter 2.1.2.2 --- The Okumura Model --- p.13 / Chapter 2.1.3 --- Indoor Propagation Models --- p.14 / Chapter 2.1.3.1 --- Alexander Distance/Power Laws --- p.14 / Chapter 2.1.3.2 --- Saleh Model --- p.15 / Chapter 2.1.3.3 --- Hashemi Experiments --- p.16 / Chapter 2.1.3.4 --- Path Loss Models --- p.17 / Chapter 2.1.3.5 --- Ray Optical Models --- p.18 / Chapter 2.2 --- Ray Tracing： Brute Force approach --- p.20 / Chapter 2.2.1 --- Physical Layout --- p.20 / Chapter 2.2.2 --- Antenna Information --- p.20 / Chapter 2.2.3 --- Source Ray Directions --- p.21 / Chapter 2.2.4 --- Formulation --- p.22 / Chapter 2.2.4.1 --- Formula of Amplitude --- p.22 / Chapter 2.2.4.2 --- Power Reference E o --- p.23 / Chapter 2.2.4.3 --- Power spreading with path length 1/d --- p.23 / Chapter 2.2.4.4 --- Antenna Patterns --- p.23 / Chapter 2.2.4.5 --- Reflection and Transmission Coefficients --- p.24 / Chapter 2.2.4.6 --- Polarization --- p.26 / Chapter 2.2.5 --- Mean Received Power --- p.26 / Chapter 2.2.6 --- Effect of Thickness --- p.27 / Chapter 2.3 --- Neural Network --- p.27 / Chapter 2.3.1 --- Architecture --- p.28 / Chapter 2.3.1.1 --- Multilayer feedforward network --- p.28 / Chapter 2.3.1.2 --- Recurrent Network --- p.29 / Chapter 2.3.1.3 --- Fuzzy ARTMAP --- p.29 / Chapter 2.3.1.4 --- Self organization map --- p.30 / Chapter 2.3.1.5 --- Modular Neural network --- p.30 / Chapter 2.3.2 --- Training Method --- p.32 / Chapter 2.3.3 --- Advantages --- p.33 / Chapter 2.3.4 --- Definition --- p.34 / Chapter 2.3.5 --- Software --- p.34 / Chapter 3 --- HYBRID MODULAR NEURAL NETWORK --- p.35 / Chapter 3.1 --- Input and Output Parameters --- p.35 / Chapter 3.2 --- Architecture --- p.36 / Chapter 3.3 --- Data Preparation --- p.42 / Chapter 3.4 --- Advantages --- p.42 / Chapter 3.5 --- Limitation --- p.43 / Chapter 3.6 --- Applicable Environment --- p.43 / Chapter 4 --- INDIVIDUAL MODULES IN HYBRID MODULAR NEURAL NETWORK --- p.45 / Chapter 4.1 --- Conversion between spherical coordinate and Cartesian coordinate --- p.46 / Chapter 4.1.1 --- Architecture --- p.46 / Chapter 4.1.2 --- Input and Output Parameters --- p.47 / Chapter 4.1.3 --- Testing result --- p.48 / Chapter 4.2 --- Performing Rotation and translation transformation --- p.53 / Chapter 4.3 --- Calculating a hit point --- p.54 / Chapter 4.3.1 --- Architecture --- p.55 / Chapter 4.3.2 --- Input and Output Parameters --- p.55 / Chapter 4.3.3 --- Testing result --- p.56 / Chapter 4.4 --- Checking if an incident ray hits a Scattering Surface --- p.59 / Chapter 4.5 --- Calculating separation distance between source point and hitting point --- p.59 / Chapter 4.5.1 --- Input and Output Parameters --- p.60 / Chapter 4.5.2 --- Data Preparation --- p.60 / Chapter 4.5.3 --- Testing result --- p.61 / Chapter 4.6 --- Calculating propagation vector of secondary ray --- p.63 / Chapter 4.7 --- Calculating polarization vector of secondary ray --- p.63 / Chapter 4.7.1 --- Architecture --- p.64 / Chapter 4.1.2 --- Input and Output Parameters --- p.65 / Chapter 4.7.3 --- Testing result --- p.68 / Chapter 4.8 --- Rejecting ray from simulation --- p.72 / Chapter 4.9 --- Calculating receiver signal --- p.73 / Chapter 4.10 --- Further comment on preparing neural network --- p.74 / Chapter 4.10.1 --- Data preparation --- p.74 / Chapter 4.10.2 --- Batch training --- p.75 / Chapter 4.10.3 --- Batch size --- p.78 / Chapter 5 --- CANONICAL EVALUATION OF MODULAR NEURAL NETWORK --- p.80 / Chapter 5.1 --- Typical environment simulation compared with ray launching --- p.80 / Chapter 5.1.1 --- Free space --- p.80 / Chapter 5.1.2 --- Metal ground reflection --- p.81 / Chapter 5.1.3 --- Dielectric ground reflection --- p.84 / Chapter 5.1.4 --- Empty Hall --- p.86 / Chapter 6 --- INDOOR PROPAGATION ENVIRONMENT APPLICATION --- p.90 / Chapter 6.1 --- Introduction --- p.90 / Chapter 6.2 --- Indoor measurement on the Third Floor of Engineering Building --- p.90 / Chapter 6.3 --- Comparison between simulation and measurement result --- p.92 / Chapter 6.3.1 --- Path 1 --- p.93 / Chapter 6.3.2 --- Path 2 --- p.95 / Chapter 6.3.3 --- Path 3 --- p.97 / Chapter 6.3.4 --- Path 4 --- p.99 / Chapter 6.3.5 --- Overall Performance --- p.100 / Chapter 6.4 --- Delay Spread Analysis --- p.101 / Chapter 6.4.1 --- Location 1 --- p.103 / Chapter 6.4.2 --- Location 2 --- p.105 / Chapter 6.4.3 --- Location 3 --- p.107 / Chapter 6.4.4 --- Location 4 --- p.109 / Chapter 6.4.5 --- Location 5 --- p.111 / Chapter 6.5 --- Summary --- p.112 / Chapter 7 --- CONCLUSION --- p.I / Chapter 7.1 --- Summary --- p.113 / Chapter 7.2 --- Recommendations for Future Work --- p.115 / PUBLICATION LIST --- p.117 / BIBLIOGRAHY --- p.118

by Siu-ming Cha. / Thesis (M.Phil.)--Chinese University of Hong Kong, 2001. / Includes bibliographical references (leaves 123-132). / Abstracts in English and Chinese. / Abstract --- p.i / Acknowledgements --- p.iv / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Objective --- p.1 / Chapter 1.2 --- Problem --- p.1 / Chapter 1.2.1 --- Motivation --- p.1 / Chapter 1.2.2 --- Approaches --- p.3 / Chapter 1.3 --- Contributions --- p.4 / Chapter 1.4 --- Organization of this Thesis --- p.5 / Chapter 2 --- Independent Component Analysis --- p.8 / Chapter 2.1 --- Overview --- p.8 / Chapter 2.2 --- The Blind Source Separation Problem --- p.8 / Chapter 2.3 --- Statistical Independence --- p.10 / Chapter 2.3.1 --- Definition --- p.10 / Chapter 2.3.2 --- Measuring Independence --- p.11 / Chapter 2.4 --- Developments of ICA Algorithms --- p.15 / Chapter 2.4.1 --- ICA Algorithm: Removal of Higher Order Dependence --- p.16 / Chapter 2.4.2 --- Assumptions in ICA Algorithms --- p.19 / Chapter 2.4.3 --- Joint Approximate Diagonalization of Eigenmatrices(JADE) --- p.20 / Chapter 2.4.4 --- Fast Fixed Point Algorithm for Independent Component Analysis(FastICA) --- p.21 / Chapter 2.5 --- Principal Component Analysis and Independent Component Anal- ysis --- p.23 / Chapter 2.5.1 --- Theoretical Comparisons between ICA and PCA --- p.23 / Chapter 2.5.2 --- Comparisons between ICA and PCA through a Simple Example --- p.24 / Chapter 2.6 --- Applications of ICA in Finance: A review --- p.27 / Chapter 2.6.1 --- Relationships between Cocktail-Party Problem and Fi- nance --- p.27 / Chapter 2.6.2 --- Security Structures Explorations --- p.28 / Chapter 2.6.3 --- Factors Interpretation and Visual Analysis --- p.29 / Chapter 2.6.4 --- Time Series Prediction by Factors --- p.29 / Chapter 2.7 --- Conclusions --- p.30 / Chapter 3 --- Factor Models in Finance --- p.31 / Chapter 3.1 --- Overview --- p.31 / Chapter 3.2 --- Factor Models and Return Generating Processes --- p.32 / Chapter 3.2.1 --- One-Factor Model --- p.33 / Chapter 3.2.2 --- Multiple-Factor Model --- p.34 / Chapter 3.3 --- Abstraction of Factor Models in Portfolio --- p.35 / Chapter 3.4 --- Typical Applications of Factor Models: Portfolio Mangement --- p.37 / Chapter 3.5 --- Different Approaches to Estimate Factor Model --- p.39 / Chapter 3.5.1 --- Time-Series Approach --- p.39 / Chapter 3.5.2 --- Cross-Section Approach --- p.40 / Chapter 3.5.3 --- Factor-Analytic Approach --- p.41 / Chapter 3.6 --- Conclusions --- p.42 / Chapter 4 --- ICA and Factor Models --- p.43 / Chapter 4.1 --- Overview --- p.43 / Chapter 4.2 --- Relationships between BSS and Factor Models --- p.43 / Chapter 4.2.1 --- Mathematical Deviation from Factor Models to Mixing Process --- p.45 / Chapter 4.3 --- Procedures of Factor Model Constructions by ICA --- p.47 / Chapter 4.4 --- Sorting Criteria for Factors --- p.48 / Chapter 4.4.1 --- Kurtosis --- p.50 / Chapter 4.4.2 --- Number of Runs --- p.52 / Chapter 4.5 --- Experiments and Results I: Factor Model Constructions --- p.53 / Chapter 4.5.1 --- Factors and their Sensitivities Extracted by ICA --- p.55 / Chapter 4.5.2 --- Factor Model Construction for a Stock --- p.60 / Chapter 4.6 --- Discussion --- p.62 / Chapter 4.6.1 --- Remarks on Applying ICA to Find Factors --- p.62 / Chapter 4.6.2 --- Independent Factors and Sparse Coding --- p.63 / Chapter 4.6.3 --- Selecting Securities for ICA --- p.63 / Chapter 4.6.4 --- Factors in Factor Models --- p.65 / Chapter 4.7 --- Conclusions --- p.66 / Chapter 5 --- Factor Model Evaluations and Selections --- p.67 / Chapter 5.1 --- Overview --- p.67 / Chapter 5.2 --- Random Residue: Requirement of Independent Factor Model --- p.68 / Chapter 5.2.1 --- Runs Test --- p.68 / Chapter 5.2.2 --- Interpretation of z-value --- p.70 / Chapter 5.3 --- Experiments and Results II: Factor Model Selections --- p.71 / Chapter 5.3.1 --- Randomness of Residues using Different Sorting Criteria --- p.71 / Chapter 5.3.2 --- Reverse Sortings of Kurtosis and Number of Runs --- p.76 / Chapter 5.4 --- Experiments and Results using FastICA --- p.80 / Chapter 5.5 --- Other Evaluation Criteria for Independent Factor Models --- p.85 / Chapter 5.5.1 --- Reconstruction Error --- p.86 / Chapter 5.5.2 --- Minimum Description Length --- p.89 / Chapter 5.6 --- Conclusions --- p.92 / Chapter 6 --- New Applications of Independent Factor Models --- p.93 / Chapter 6.1 --- Overview --- p.93 / Chapter 6.2 --- Applications to Financial Trading System --- p.93 / Chapter 6.2.1 --- Modifying Shocks in Stocks --- p.96 / Chapter 6.2.2 --- Modifying Sensitivity to Residue --- p.100 / Chapter 6.3 --- Maximization of Higher Moment Utility Function --- p.104 / Chapter 6.3.1 --- No Good Approximation to Utility Function --- p.107 / Chapter 6.3.2 --- Uncorrelated and Independent Factors in Utility Ma mizationxi- --- p.108 / Chapter 6.4 --- Conclusions --- p.110 / Chapter 7 --- Future Works --- p.111 / Chapter 8 --- Conclusion --- p.113 / Chapter A --- Stocks used in experiments --- p.116 / Chapter B --- Proof for independent factors outperform dependent factors in prediction --- p.117 / Chapter C --- Demixing Matrix and Mixing Matrix Found by JADE --- p.119 / Chapter D --- Moments and Cumulants --- p.120 / Chapter D.1 --- Moments --- p.120 / Chapter D.2 --- Cumulants --- p.121 / Chapter D.3 --- Cross-Cumulants --- p.121 / Bibliography --- p.123

Orientador: Paulo Roberto de Aguiar / Banca: Eduardo Carlos Bianchi / Banca: Janaina Fracaro de Souza Gonçalves / Resumo: O monitoramento em tempo real do processo de dressagem vem se tornando cada vez mais necessário, pois tem um papel muito importante no acabamento de peças produzidas pelo processo de retificação. Por outro lado, o desgaste dos dressadoers é muito custoso e pouco eficiente para ser monitorado visualmente, como normalmente é feito nas indústrias. O sensor de vibração é uma grande ferramenta na automação desse processo, porém ainda é pouco utilizado, como se constata na literatura. Este trabalho apresenta um método de classificação do desgaste da ferramenta de ponta única em três condições distintas (novo, meia-vida e desgastado), por meio de vibração e redes neurais. Ensaios de dressagens foram realizados em uma retificadora plana tangencial, rebolo de óxido de alumínio, com a aquisição dos sinais de vibração por meio de um sensor fixo no suporte do dressador. Um estudo foi desenvolvido do espectro do sinal para as três condições de desgaste, no qual sete bandas de frequências foram selecionadas. Vários modelos neurais foram testados, os quais possuíam como entradas duas estatísticas obtidas a partir do sinal original filtrado para uma dada banda de frequência selecionada. Após centenas de combinações de entradas, número de camadas ocultas e número de neurônio, dois melhores modelos foram escolhidos e analisados, os quais apresentaram resultados com até 98,3% de taxa de acertos / Abstract: Real time monitoring of the dressing process is becoming more and more necessary because it plays a very important role in the finish of the part manufactured by the grinding process. On the other hand, dresser wear is very expensive and not much effective to be monitored visually, but it is usually so developed in industry. The vibration sensor can be a useful tool in the process automation; however, it is rarely used as can be verified in research works. This work presents a classification method for three wear conditions (new, semi-new, and worn) of single-point dresser by using vibration signal and neural networks. Experimental runs were carried out in a surface grinding machine equipped with aluminium oxide grinding wheel, where the vibration signal was acquired by a fixed sensor attached to the dresser bolder. The signal spectra analysis was performed with regarding to the aforementioned wear conditions, and seven frequency bands were selected. Several neural network models were tested, which had two input statistics from the digital processing of the raw signal filtered for a given frequency band selected. Following hundreds of input combinations, number of hidder layers and neurons, two best models were chosen and analyzed, which showed results with up to 98.3% success rate / Mestre

by Zhi-Bin Lai. / Thesis submitted in: December 1998. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1999. / Includes bibliographical references (leaves 111-116). / Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- ICA Model --- p.1 / Chapter 1.2 --- ICA Algorithms --- p.3 / Chapter 1.3 --- Foreign Exchange Rate Scheme --- p.9 / Chapter 1.4 --- Problem Motivation --- p.10 / Chapter 1.5 --- Main Contribution of the Thesis --- p.10 / Chapter 1.6 --- Other Contribution of the Thesis --- p.11 / Chapter 1.7 --- Organization of the Thesis --- p.11 / Chapter 2 --- Heuristic Dominant ICs Sorting --- p.13 / Chapter 2.1 --- L1 Norm Sorting --- p.13 / Chapter 2.2 --- Lp Norm (L3 Norm) Sorting --- p.14 / Chapter 2.3 --- Problem Motivation --- p.15 / Chapter 2.4 --- Determination of Dominant ICs --- p.15 / Chapter 2.5 --- ICA in Foreign Exchange Rate Markets --- p.16 / Chapter 2.6 --- Comparison of Two Heuristic Methods --- p.16 / Chapter 2.6.1 --- Experiment 1: US Dollar vs Swiss Franc --- p.18 / Chapter 2.6.2 --- Experiment 2: US Dollar vs Australian Dollar --- p.21 / Chapter 2.6.3 --- Experiment 3: US Dollar vs Canadian Dollar --- p.24 / Chapter 2.6.4 --- Experiment 4: US Dollar vs French Franc --- p.27 / Chapter 3 --- Forward Selection under MSE Measurement --- p.30 / Chapter 3.1 --- Order-Sorting Criterion --- p.30 / Chapter 3.2 --- Order Sorting Approaches --- p.30 / Chapter 3.3 --- Forward Selection Approach --- p.31 / Chapter 3.4 --- Comparison of Three Dominant ICs Sorting Methods --- p.32 / Chapter 3.4.1 --- Experiment 1: US Dollar vs Swiss Franc --- p.33 / Chapter 3.4.2 --- Experiment 2: US Dollar vs Australian Dollar --- p.37 / Chapter 3.4.3 --- Experiment 3: US Dollar vs Canadian Dollar --- p.41 / Chapter 3.4.4 --- Experiment 4: US Dollar vs French Franc --- p.45 / Chapter 4 --- Backward Elimination Tendency Error --- p.49 / Chapter 4.1 --- Tendency Error Scheme --- p.49 / Chapter 4.2 --- Order-Sorting Criterion --- p.50 / Chapter 4.3 --- Order Sorting Approaches --- p.50 / Chapter 4.4 --- Backward Elimination Tendency Error Approach --- p.51 / Chapter 4.5 --- Determination of Dominant ICs --- p.52 / Chapter 4.6 --- Comparison Between Three Approaches --- p.53 / Chapter 4.6.1 --- Experiment Results on USD-SWF Return --- p.53 / Chapter 4.6.2 --- Experiment Results on USD-AUD Return --- p.57 / Chapter 4.6.3 --- Experiment Results on USD-CAD Return --- p.61 / Chapter 4.6.4 --- Experiment Results on USD-FRN Return --- p.65 / Chapter 5 --- Other Analysis of ICA in Foreign Exchange Rate Markets --- p.69 / Chapter 5.1 --- Variance Characteristics of ICs and PCs --- p.69 / Chapter 5.2 --- Reconstruction Ability between PCA and ICA --- p.70 / Chapter 5.3 --- Properties of Independent Components --- p.70 / Chapter 5.4 --- Autocorrelation --- p.73 / Chapter 5.5 --- Rescaled Analysis --- p.73 / Chapter 6 --- Conclusion and Further Work - --- p.78 / Chapter 6.1 --- Conclusion --- p.78 / Chapter 6.2 --- Further Work --- p.79 / Chapter A --- Fast Implement of LPM Algorithm --- p.80 / Chapter A.1 --- Review of Selecting Subsets from Regression Variables --- p.80 / Chapter A.2 --- Unconstrained Gradient Based Optimization Methods Survey --- p.85 / Chapter A.3 --- Characteristics of the Original LPM Algorithm --- p.88 / Chapter A.4 --- Constrained Learning Rate Adaptation Method --- p.89 / Chapter A.5 --- Gradient Descent with Momentum Method --- p.98

Despite considerable progress in mapping the fly’s connectome and elucidating the patterns of information flow in its brain, the complexity of the fly brain’s structure and the still-incomplete state of knowledge regarding its neural circuitry pose significant challenges beyond satisfying the computational resource requirements of current fly brain models that must be addressed to successfully reverse the information processing capabilities of the fly brain. These include the need to explicitly facilitate collaborative development of brain models by combining the efforts of multiple researchers, and the need to enable programmatic generation of brain models that effectively utilize the burgeoning amount of increasingly detailed publicly available fly connectome data. This thesis presents an open pipeline for modular construction of executable models of the fruit fly brain from incomplete biological brain data that addresses both of the above requirements. This pipeline consists of two major open-source components respectively called Neurokernel and NeuroArch. Neurokernel is a framework for collaborative construction of executable connectome-based fly brain models by integration of independently developed models of different functional units in the brain into a single emulation that can be executed upon multiple Graphics Processing Units (GPUs). Neurokernel enforces a programming model that enables functional unit models that comply with its interface requirements to communicate during execution regardless of their internal design. We demonstrate the power of this programming model by using it to integrate independently developed models of the fly retina and lamina into a single vision processing system. We also show how Neurokernel’s communication performance can scale over multiple GPUs, number of functional units in a brain emulation, and over the number of communication ports exposed by a functional unit model. Although the increasing amount of experimentally obtained biological data regarding the fruit fly brain affords brain modelers a potentially valuable resource for model development, the actual use of this data to construct executable neural circuit models is currently challenging because of the disparate nature of different data sources, the range of storage formats they use, and the limited query features of those formats complicates the process of inferring executable circuit designs from biological data. To overcome these limitations, we created a software package called NeuroArch that defines a data model for concurrent representation of both biological data and model structure and the relationships between them within a single graph database. Coupled with a powerful interface for querying both types of data within the database in a uniform high-level manner, this representation enables construction and dispatching of executable neural circuits to Neurokernel for execution and evaluation. We demonstrate the utility of the NeuroArch/Neurokernel pipeline by using the packages to generate an executable model of the central complex of the fruit fly brain from both published and hypothetical data regarding overlapping neuron arborizations in different regions of the central complex neuropils. We also show how the pipeline empowers circuit model designers to devise computational analogues to biological experiments such as parallel concurrent recording from multiple neurons and emulation of genetic mutations that alter the fly’s neural circuitry.

Sequences of feature vectors are a natural way of representing temporal data. Given a database of sequences, a fundamental task is to find the database entry which is the most similar to a query. In this thesis, we present learning-based methods for efficiently and accurately comparing sequences in order to facilitate large-scale sequence search. Throughout, we will focus on the problem of matching MIDI files (a digital score format) to a large collection of audio recordings of music. The combination of our proposed approaches enables us to create the largest corpus of paired MIDI files and audio recordings ever assembled. Dynamic time warping (DTW) has proven to be an extremely effective method for both aligning and matching sequences. However, its performance is heavily affected by factors such as the feature representation used and its adjustable parameters. We therefore investigate automatically optimizing DTW-based alignment and matching of MIDI and audio data. Our approach uses Bayesian optimization to tune system design and parameters over a synthetically-created dataset of audio and MIDI pairs. We then perform an exhaustive search over DTW score normalization techniques to find the optimal method for reporting a reliable alignment confidence score, as required in matching tasks. This results in a DTW-based system which is conceptually simple and highly accurate at both alignment and matching. We also verify that this system achieves high performance in a large-scale qualitative evaluation of real-world alignments. Unfortunately, DTW can be far too inefficient for large-scale search when sequences are very long and consist of high-dimensional feature vectors. We therefore propose a method for mapping sequences of continuously-valued feature vectors to downsampled sequences of binary vectors. Our approach involves training a pair of convolutional networks to map paired groups of subsequent feature vectors to a Hamming space where similarity is preserved. Evaluated on the task of matching MIDI files to a large database of audio recordings, we show that this technique enables 99.99\% of the database to be discarded with a modest false reject rate while only requiring 0.2\% of the time to compute. Even when sped-up with a more efficient representation, the quadratic complexity of DTW greatly hinders its feasibility for very large-scale search. This cost can be avoided by mapping entire sequences to fixed-length vectors in an embedded space where sequence similarity is approximated by Euclidean distance. To achieve this embedding, we propose a feed-forward attention-based neural network model which can integrate arbitrarily long sequences. We show that this approach can extremely efficiently prune 90\% of our audio recording database with high confidence. After developing these approaches, we applied them together to the practical task of matching 178,561 unique MIDI files to the Million Song Dataset. The resulting ``Lakh MIDI Dataset'' provides a potential bounty of ground truth information for audio content-based music information retrieval. This can include transcription, meter, lyrics, and high-level musicological features. The reliability of the resulting annotations depends both on the quality of the transcription and the accuracy of the score-to-audio alignment. We therefore establish a baseline of reliability for score-derived information for different content-based MIR tasks. Finally, we discuss potential future uses of our dataset and the learning-based sequence comparison methods we developed.

Since the approval of the first therapeutic monoclonal antibody in 1986, monoclonal antibody has become an important class of drugs within the biopharmaceutical industry, with indications and superior efficacy across multiple therapeutic areas, such as oncology and immunology. Although there has been great advance in this field, there are still challenges that hinder or delay the development and approval of new antibodies. For example, we have seen issues in manufacturing, such as quality, process inconsistency and large manufacturing cost, which can be attributed to production failure, delay in approval and drug shortage. Recently, the development of new technologies, such as Process Analytical Tools (PCT), and the use of statistical tools, such as quality by design (QbD), Design of Experiment (DoE) and Statistical Process Control (SPC), has enabled us to identify critical process parameters and attributes, and monitor manufacturing performance. However, these methods might not be reliable or comprehensive enough to accurately describe the relationship between critical process parameters and attributes, or still lack the ability to forecast manufacturing performance. In this work, by utilizing multiple modeling approaches, we have developed a systematic framework to optimize and control monoclonal antibody manufacturing process. In our first study, we leverage DoE-PCA approach to unambiguously identify critical process parameters to improve process yield and cost of goods, followed by the use of Monte Carlo simulation to validate the impact of parameters on these attributes. In our second study, we use a Bayesian approach to predict product quality for future manufacturing batches, and hence mitigation strategies can be put in place if the data suggest a potential deviation. Finally, we use neural network model to accurately characterize the impurity reduction of each purification step, and ultimately use this model to develop acceptance criteria for the feed based on the predetermined process specifications. Overall, the work in this thesis demonstrates that the framework is powerful and more reliable for process optimization, monitoring and control.

There are more neurons in the human brain than seconds in a lifetime. Given this incredible number how can we hope to understand the computations carried out by the full ensemble of neural firing patterns? And neural activity is not the only substrate available for computations. The incredible diversity of function found within biological organisms is matched by an equally rich reservoir available for computation. If we are interested in the metamorphosis of a caterpillar to a butterfly we could explore how DNA expression changes the cell. If we are interested in developing therapeutic drugs we could explore receptors and ion channels. And if we are interested in how humans and other animals interpret incoming streams of sensory information and process them to make moment-by-moment decisions then perhaps we can understand much of this behavior by studying the firing rates of neurons. This is the level and approach we will take in this thesis. Given this diversity of potential reservoirs for computation, combined with limitations in recording technologies, it can be difficult to satisfactorily conclude that we are studying the full set of neural dynamics involved in a particular task. To overcome this limitation, we augment the study of neural activity with the study of artificial recurrent neural networks (RNNs) trained to mimic the behavior of humans and other animals performing experimental tasks. The inputs to the RNN are time-varying signals representing experimental stimuli and we adjust the parameters of the RNN so its time-varying outputs are the desired behavioral responses. In these artificial RNNs we have complete information about the network connectivity and moment-by-moment firing patterns and know, by design, that these are the only computational mechanisms being used to solve the tasks. If the artificial RNN and electrode recordings of real neurons have the same dynamics we can be more confident that we are studying the sufficient set of biological dynamics involved in the task. This is important if we want to make claims about the types of dynamics required, and observed, for various computational tasks, as is the case in Chapter 2 of this thesis. In Chapter 2 we develop tests to identify several classes of neural dynamics. The specific neural dynamic regimes we focus on are interesting because they each have different computational capabilities, including, the ability to keep track of time, or preserve information robustly against the flow of time (working memory). We then apply these tests to electrode recordings from nonhuman primates and artificial RNNs to understand how neural networks are able to simultaneously keep track of time and remember previous experiences in working memory. To accomplish both computational goals the brain is thought to use distinct neural dynamics; stable neural trajectories can be used as a clock to coordinate cognitive activity whereas attractor dynamics provide a stable mechanism for memory storage but all timing information is lost. To identify these neural regimes we decode the passage of time from neural data. Additionally, to encode the passage of time, stabilized neural trajectories can be either high-dimensional as is the case for randomly connected recurrent networks (chaotic reservoir networks) or low-dimensional as is the case for artificial RNNs trained with backpropagation through time. To disambiguate these models we compute the cumulative dimensionality of the neural trajectory as it evolves over time. Recurrent neural networks can also be used to generate hypotheses about neural computation. In Chapter 3 we use RNNs to generate hypotheses about the diverse set of neural response properties seen during spatial navigation, in particular, grid cells, and other spatial correlates, including border cells and band-like cells. The approach we take is 1) pick a task that requires navigation (spatial or mental), 2) create a RNN to solve the task, and 3) adjust the task or constraints on the neural network such that grid cells and other spatial response patterns emerge naturally as the network learns to perform the task. We trained RNNs to perform navigation tasks in 2D arenas based on velocity inputs. We find that grid-like spatial response patterns emerge in trained networks, along with units that exhibit other spatial correlates, including border cells and band-like cells. Surprisingly, the order of the emergence of grid-like and border cells during network training is also consistent with observations from developmental studies. Together, our results suggest that grid cells, border cells and other spatial correlates observed in the Entorhinal Cortex of the mammalian brain may be a natural solution for representing space efficiently given the predominant recurrent connections in the neural circuits. All the tasks we have considered so far in this thesis require memory, but in Chapter 4 we explicitly explore the interactions between multiple memories in a recurrent neural network. Memory is the hallmark of recurrent neural networks, in contrast to standard feedforward neural networks where all signals travel in one direction from inputs to outputs and the network contains no memory of previous experiences. A recurrent neural network, as the name suggests, contains feedback loops giving the network the computational power of memory. In this chapter we train a RNN to perform a human psychophysics experiment and find that in order to reproduce human behavior, noise must be added to the network, causing the RNN to use more stable discrete memories to constrain less stable continuous memories.

In practice, the application and extension of the ICA model depend on the problem and the data to be investigated. We finally focus on GARCH models in finance, and show that estimation of univariate or multivariate GARCH models is actually a nonlinear ICA problem; maximizing the likelihood is equivalent to minimizing the statistical dependence in standardized residuals. ICA can then be used for factor extraction in multivariate factor GARCH models. We also develop some extensions of ICA for this task. These techniques for extracting factors from multivariate return series are compared both theoretically and experimentally. We find that the one based on conditional decorrelation between factors behaves best. / In this thesis, first we consider the problem of source separation of post-nonlinear (PNL) mixtures, which is an extension of ICA to the nonlinear mixing case. With a large number of parameters, existing methods are computation-demanding and may be prone to local optima. Based on the fact that linear mixtures of independent variables tend to be Gaussian, we develop a simple and efficient method for this problem, namely extended Gaussianization. With Gaussianization as preprocessing, this method approximates each linear mixture of independent sources by the Cornish-Fisher expansion with only two parameters. Inspired by the relationship between the PNL mixing model and the Wiener system, extended Gaussianization is also proposed for blind inversion of Wiener systems. / Independent component analysis (ICA) is a recent and powerful technique for recovering latent independent sources given only their mixtures. The basic ICA model assumes that sources are linearly mixed and mutually independent. / Next, we study the subband decomposition ICA (SDICA) model, which extends the basic ICA model to allow dependence between sources by assuming that only some narrow-band source sub-components are independent. In SDICA, it is difficult to determine the subbands of source independent sub-components. We discuss the feasibility of performing SDICA in an adaptive manner. An adaptive method, called band selective ICA, is then proposed for this task. We also investigate the relationship between overcomplete ICA and SDICA and show that band selective ICA can solve the overcomplete ICA problems with sources having specific frequency localizations. Experimental results on separating images of human faces as well as artificial data are presented to verify the powerfulness of band selective ICA. / Zhang Kun. / "July 2005." / Adviser: Lai-Wan Chan. / Source: Dissertation Abstracts International, Volume: 67-07, Section: B, page: 3925. / Thesis (Ph.D.)--Chinese University of Hong Kong, 2005. / Includes bibliographical references (p. 218-234). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. [Ann Arbor, MI] : ProQuest Information and Learning, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Abstract in English and Chinese. / School code: 1307.

by Ka Keung Caramon Lee. / "March 29, 2004." / Thesis (Ph.D.)--Chinese University of Hong Kong, 2004. / Includes bibliographical references (p. 188-210). / Electronic reproduction. Hong Kong : Chinese University of Hong Kong, [2012] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Electronic reproduction. Ann Arbor, MI : ProQuest Information and Learning Company, [200-] System requirements: Adobe Acrobat Reader. Available via World Wide Web. / Mode of access: World Wide Web. / Abstracts in English and Chinese.