Deep neural networks (DNNs), particularly convolutional neural networks (CNNs), have been widely applied in the many fields, such as computer vision, natural language processing, speech recognition and etc. Although DNNs achieve dramatic accuracy improvements in these real-world tasks, they require significant amounts of resources (e.g., memory, energy, storage, bandwidth and computation resources). This limits the application of these networks on resource-constrained systems, such as mobile and edge devices. A large body of literature has been proposed to addresses this problem from the perspective of compressing DNNs while preserving their performance. In this thesis, we focus on compressing deep CNNs based on vector quantization techniques. The first part of this thesis summarizes some basic concepts in machine learning and popular techniques on model compression, including pruning, quantization, low-rank factorization and knowledge distillation approaches. Our main interest is quantization techniques, which compress networks by reducing the precision of parameters. Full-precision weights, activations and even gradients in networks can be quantized to 16-bit floating point numbers, 8-bit integers, or even binary numbers. Despite a possible performance degradation, quantization can greatly reduce the model size while maintaining model accuracy. In the second part of this thesis, we propose a novel vector quantization approach, which we refer to as Vector Quantization with Learned Codebook, or VQLC, for CNNs. Rather than performing scalar quantization, we choose vector quantization that can simultaneously quantize multiple weights at once. Instead of taking a pretraining/clustering approach as in most works, in VQLC, the codebook for quantization are learned together with neural network training from scratch. For the forward pass, the traditional convolutional filters are replaced by the convex combinations of a set of learnable codewords. During inference, the compressed model will be represented by a small-sized codebook and a set of indices, resulting in a significant reduction of model size while preserving the network's performance. Lastly, we validate our approach by quantizing multiple modern CNNs on several popular image classification benchmarks and compare with state-of-the-art quantization techniques. Our experimental results show that VQLC demonstrates at least comparable and often superior performance to the existing schemes. In particular, VQLC demonstrates significant advantages over the existing approaches on wide networks at the high rate of compression.

The major results of this work concern perfect ideals of ordered vector spaces, and a representation theory for ordered vector spaces. Perfect ideals are characterized by the property that their annihilators in the order dual are ideals. We obtain a number of conditions for an ordered vector space which are equivalent to the intersection of the set of perfect maximal ideals being 0. We also obtain conditions which permit an ordered vector space to be represented as a subspace of the sections of a vector bundle. This generalizes the representation theory for odered vector spaces with unit. / Thesis / Doctor of Philosophy (PhD)

No description available.

No description available.

No description available.

No description available.

No description available.

No description available.

In this thesis, a support vector machine (SVM) is used to develop a model to predict Arias Intensity. Arias Intensity is a measure of the strength of ground motions that considers both the amplitude and the duration of ground motions. In this research, a subset of the database from the “Next Generation and the duration of Ground-Motion Attenuation Models” project was used as the training data. The data includes 3525 ground motion records from 175 earthquakes. This research provides the assessment of historical earthquakes using arias intensity data. Support vector machine uses a Kernel function to transform the data into a high dimensional space where relationships between the variables can be efficiently described using simpler models. In this research, after testing several kernel functions, a Gaussian Kernel was selected for the predictive model. The resulting model uses magnitude, epicentral distance, and the shear wave velocity as the predictor of Arias Intensity.

This thesis studies the problem of frequentist model averaging over a set of multiple $\epsilon$-support vector regression (SVR) models, where the support vector machine (SVM) algorithm was extended to function estimation involving continuous targets, instead of categorical ones. By assigning weights to a set of candidate models instead of selecting the least misspecified one, model averaging presents a strong alternative to model selection for tackling model uncertainty. Not only do we describe the construction of smoothed BIC/AIC model averaging weights, but we also propose a Mallows model averaging procedure which selects model weights by minimizing Mallows' criterion. We conduct two studies where the set of candidate models can either include or not include the true model by making use of simulated random samples obtained from different data-generating processes of analytic form. In terms of mean squared error, we demonstrate that our proposed method outperforms other model averaging and model selection methods that were tested, and the gain is more substantial for smaller sample sizes with larger signal-to-noise ratios. / Thesis / Master of Science (MSc)