Master of Science / Department of Statistics / Nora Bello / Ordered categorical responses are frequently encountered in many disciplines. Examples of interest in agriculture include quality assessments, such as for soil or food products, and evaluation of lesion severity, such as teat ends status in dairy cattle. Ordered categorical responses are characterized by multiple categories or levels recorded on a ranked scale that, while apprising relative order, are not informative of magnitude of or proportionality between levels. A number of statistically sound models for ordered categorical responses have been proposed, such as logistic regression and probit models, but these are commonly underutilized in practice. Instead, the ordinary least squares linear regression model is often employed with ordered categorical responses despite violation of basic model assumptions. In this study, the inferential implications of this approach are investigated using a simulation study that evaluates robustness based on realized Type I error rate and statistical power. The design of the simulation study is motivated by applied research cases reported in the literature. A variety of plausible scenarios were considered for simulation, including various shapes of the frequency distribution and different number of categories of the ordered categorical response. Using a real dataset on frequency of antimicrobial use in feedlots, I demonstrate the inferential performance of ordinary least squares linear regression on ordered categorical responses relative to a probit model.

This master thesis will investigate one solution to the problem issues with nested stochastic simulation arising when the future value of a portfolio need to be calculated. The solution investigated is the Least-squares Monte-Carlo method, where regression is used to obtain a proxy function for the given portfolio value. We will further investigate how to generate an optimal regression function that minimizes the number of terms in the regression function and reduces the risk of overtting the regression.

Numerical simulations are presented for three dimensional viscous incompressible free shear flows. The numerical method is based on solving the vorticity equation using Vortex-In-Cell method. In this method, the vorticity field is discretized into a finite set of Lagrangian elements (particles) and the computational domain is covered by Eulerian mesh. Velocity field is computed on the mesh by solving Poisson equation. The solution proceeds in time by advecting the particles with the flow. Second order Adam-Bashford method is used for time integration. Exchange of information between Lagrangian particles and Eulerian grid is carried out using the M’4 interpolation scheme. The classical inviscid scheme is enhanced to account for stretching and viscous effects. For that matter, two schemes are used. The first one used periodic remeshing of the vortex particles along with fourth order finite difference approximation for the partial derivatives of the stretching and viscous terms. In the second scheme, derivatives are approximated by least squares polynomial. The novelty of this work is signified by using the moving least squares technique within the framework of the Vortex-in-Cell method and implementing it to a three dimensional temporal mixing layer. Comparisons of the mean flow and velocity statistics are made with experimental studies. The results confirm the validity of the present schemes. Both schemes also demonstrate capability to qualitatively capture significant flow scales, and allow gaining physical insight as to the development of instabilities and the formation of three dimensional vortex structures. The two schemes show acceptable low numerical diffusion as well.

Before raw seismic data records are interpretable by geologists, geophysicists must process these data using a technique called migration. Migration spatially repositions the acoustic energy in a seismic record to its correct location in the subsurface. Traditional migration techniques used a transpose approximation to a true acoustic propagation operator. Conventional least squares migration uses a true inverse operator, but is limited in functionality by the large size of modern seismic datasets. This research uses a new technique, called stabilized least squares migration, to correctly migrate seismic data records using a true inverse operator. Contrary to conventional least squares migration, this new technique allows for errors over ten percent in the underlying subsurface velocity model, which is a large limitation in conventional least squares migration. The stabilized least squares migration also decreases the number of iterations required by conventional least squares migration algorithms by an average of about three iterations on the sample data tested in this research.

This paper demonstrates that even regression results obtained by techniques close to the standard ordinary least squares (OLS) method can be difficult to replicate if a stochastic model fitting algorithm is employed. / Series: Research Report Series / Department of Statistics and Mathematics

This project analyzed the giving data of Worcester Polytechnic Institute's alumni and other constituents (parents, friends, neighbors, etc.) from fiscal year 1983 to 2007 using a two-stage modeling approach. Logistic regression analysis was conducted in the first stage to predict the likelihood of giving for each constituent, followed by linear regression method in the second stage which was used to predict the amount of contribution to be expected from each contributor. Box-Cox transformation was performed in the linear regression phase to ensure the assumption underlying the model holds. Due to the nature of the data, multiple imputation was performed on the missing information to validate generalization of the models to a broader population. Concepts from the field of direct and database marketing, like "score" and "lift", were also introduced in this report.

by Wan, Kwok Fai. / Thesis (Ph.D.)--Chinese University of Hong Kong, 1995. / Includes bibliographical references (leaves 138-148). / Chapter Chapter 1. --- Adaptive signal processing and its applications --- p.1 / Chapter 1.1. --- Introduction --- p.1 / Chapter 1.2. --- Applications of adaptive system --- p.3 / Chapter 1.2.1. --- Adaptive noise cancellation --- p.3 / Chapter 1.2.2. --- Adaptive echo cancellation --- p.5 / Chapter 1.2.3. --- Adaptive line enhancement --- p.5 / Chapter 1.2.4. --- Adaptive linear prediction --- p.7 / Chapter 1.2.5. --- Adaptive system identification --- p.8 / Chapter 1.3. --- Algorithms for adaptive systems --- p.10 / Chapter 1.4. --- Transform domain adaptive filtering --- p.12 / Chapter 1.5 --- The motivation and organization of the thesis --- p.13 / Chapter Chapter 2. --- Time domain split-path adaptive filter --- p.16 / Chapter 2.1. --- Adaptive transversal filter and the LMS algorithm --- p.17 / Chapter 2.1.1. --- Wiener-Hopf solution --- p.17 / Chapter 2.1.2. --- The LMS adaptive algorithm --- p.20 / Chapter 2.2. --- Split structure adaptive filtering --- p.23 / Chapter 2.2.1. --- Split structure of an adaptive filter --- p.24 / Chapter 2.2.2. --- Split-path structure for a non-symmetric adaptive filter --- p.25 / Chapter 2.3. --- Split-path adaptive median filtering --- p.29 / Chapter 2.3.1. --- Median filtering and median LMS algorithm --- p.29 / Chapter 2.3.2. --- The split-path median LMS (SPMLMS) algorithm --- p.32 / Chapter 2.3.3. --- Convergence analysis of SPMLMS --- p.36 / Chapter 2.4. --- Computer simulation examples --- p.41 / Chapter 2.5. --- Summary --- p.45 / Chapter Chapter 3. --- Multi-stage split structure adaptive filtering --- p.46 / Chapter 3.1. --- Introduction --- p.46 / Chapter 3.2. --- Split structure for a symmetric or an anti-symmetric adaptive filter --- p.48 / Chapter 3.3. --- Multi-stage split structure for an FIR adaptive filter --- p.56 / Chapter 3.4. --- Properties of the split structure LMS algorithm --- p.59 / Chapter 3.5. --- Full split-path adaptive algorithm for system identification --- p.66 / Chapter 3.6. --- Summary --- p.71 / Chapter Chapter 4. --- Transform domain split-path adaptive algorithms --- p.72 / Chapter 4.1. --- Introduction --- p.73 / Chapter 4.2. --- general description of transforms --- p.74 / Chapter 4.2.1. --- Fast Karhunen-Loeve transform --- p.75 / Chapter 4.2.2. --- Symmetric cosine transform --- p.77 / Chapter 4.2.3. --- Discrete sine transform --- p.77 / Chapter 4.2.4. --- Discrete cosine transform --- p.78 / Chapter 4.2.5. --- Discrete Hartley transform --- p.78 / Chapter 4.2.6. --- Discrete Walsh transform --- p.79 / Chapter 4.3. --- Transform domain adaptive filters --- p.80 / Chapter 4.3.1. --- Structure of transform domain adaptive filters --- p.80 / Chapter 4.3.2. --- Properties of transform domain adaptive filters --- p.83 / Chapter 4.4. --- Transform domain split-path LMS adaptive predictor --- p.84 / Chapter 4.5. --- Performance analysis of the TRSPAF --- p.93 / Chapter 4.5.1. --- Optimum Wiener solution --- p.93 / Chapter 4.5.2. --- Steady state MSE and convergence speed --- p.94 / Chapter 4.6. --- Computer simulation examples --- p.96 / Chapter 4.7. --- Summary --- p.100 / Chapter Chapter 5. --- Tracking optimal convergence factor for transform domain split-path adaptive algorithm --- p.101 / Chapter 5.1. --- Introduction --- p.102 / Chapter 5.2. --- The optimal convergence factors of TRSPAF --- p.104 / Chapter 5.3. --- Tracking optimal convergence factors for TRSPAF --- p.110 / Chapter 5.3.1. --- Tracking optimal convergence factor for gradient-based algorithms --- p.111 / Chapter 5.3.2. --- Tracking optimal convergence factors for LMS algorithm --- p.112 / Chapter 5.4. --- Comparison of optimal convergence factor tracking method with self-orthogonalizing method --- p.114 / Chapter 5.5. --- Computer simulation results --- p.116 / Chapter 5.6. --- Summary --- p.121 / Chapter Chapter 6. --- A unification between split-path adaptive filtering and discrete Walsh transform adaptation --- p.122 / Chapter 6.1. --- Introduction --- p.122 / Chapter 6.2. --- A new ordering of the Walsh functions --- p.124 / Chapter 6.3. --- Relationship between SM-ordered Walsh function and other Walsh functions --- p.126 / Chapter 6.4. --- Computer simulation results --- p.132 / Chapter 6.5. --- Summary --- p.134 / Chapter Chapter 7. --- Conclusion --- p.135 / References --- p.138

by Chi-cheong Szeto. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1999. / Includes bibliographical references (leaves 162-165). / Abstracts in English and Chinese. / Abstract --- p.i / Acknowledgment --- p.ii / Table of Contents --- p.iii / Chapter Chapter 1 --- Introduction --- p.1 / Chapter 1.1 --- Time series prediction --- p.1 / Chapter 1.2 --- Forecasting models --- p.1 / Chapter 1.2.1 --- Networks using time delays --- p.2 / Chapter 1.2.1.1 --- Model description --- p.2 / Chapter 1.2.1.2 --- Limitation --- p.3 / Chapter 1.2.2 --- Networks using context units --- p.3 / Chapter 1.2.2.1 --- Model description --- p.3 / Chapter 1.2.2.2 --- Limitation --- p.6 / Chapter 1.2.3 --- Layered fully recurrent networks --- p.6 / Chapter 1.2.3.1 --- Model description --- p.6 / Chapter 1.2.3.2 --- Our selection and motivation --- p.8 / Chapter 1.2.4 --- Other models --- p.8 / Chapter 1.3 --- Learning methods --- p.8 / Chapter 1.3.1 --- First order and second order methods --- p.9 / Chapter 1.3.2 --- Nonlinear least squares methods --- p.11 / Chapter 1.3.2.1 --- Levenberg-Marquardt method ´ؤ our selection and motivation --- p.13 / Chapter 1.3.2.2 --- Levenberg-Marquardt method - algorithm --- p.13 / Chapter 1.3.3 --- "Batch mode, semi-sequential mode and sequential mode of updating" --- p.15 / Chapter 1.4 --- Jacobian matrix calculations in recurrent networks --- p.15 / Chapter 1.4.1 --- RTBPTT-like Jacobian matrix calculation --- p.15 / Chapter 1.4.2 --- RTRL-like Jacobian matrix calculation --- p.17 / Chapter 1.4.3 --- Comparison between RTBPTT-like and RTRL-like calculations --- p.18 / Chapter 1.5 --- Computation complexity reduction techniques in recurrent networks --- p.19 / Chapter 1.5.1 --- Architectural approach --- p.19 / Chapter 1.5.1.1 --- Recurrent connection reduction method --- p.20 / Chapter 1.5.1.2 --- Treating the feedback signals as additional inputs method --- p.20 / Chapter 1.5.1.3 --- Growing network method --- p.21 / Chapter 1.5.2 --- Algorithmic approach --- p.21 / Chapter 1.5.2.1 --- History cutoff method --- p.21 / Chapter 1.5.2.2 --- Changing the updating frequency from sequential mode to semi- sequential mode method --- p.22 / Chapter 1.6 --- Motivation for using block-diagonal Hessian matrix --- p.22 / Chapter 1.7 --- Objective --- p.23 / Chapter 1.8 --- Organization of the thesis --- p.24 / Chapter Chapter 2 --- Learning with the block-diagonal Hessian matrix --- p.25 / Chapter 2.1 --- Introduction --- p.25 / Chapter 2.2 --- General form and factors of block-diagonal Hessian matrices --- p.25 / Chapter 2.2.1 --- General form of block-diagonal Hessian matrices --- p.25 / Chapter 2.2.2 --- Factors of block-diagonal Hessian matrices --- p.27 / Chapter 2.3 --- Four particular block-diagonal Hessian matrices --- p.28 / Chapter 2.3.1 --- Correlation block-diagonal Hessian matrix --- p.29 / Chapter 2.3.2 --- One-unit block-diagonal Hessian matrix --- p.35 / Chapter 2.3.3 --- Sub-network block-diagonal Hessian matrix --- p.35 / Chapter 2.3.4 --- Layer block-diagonal Hessian matrix --- p.36 / Chapter 2.4 --- Updating methods --- p.40 / Chapter Chapter 3 --- Data set and setup of experiments --- p.41 / Chapter 3.1 --- Introduction --- p.41 / Chapter 3.2 --- Data set --- p.41 / Chapter 3.2.1 --- Single sine --- p.41 / Chapter 3.2.2 --- Composite sine --- p.42 / Chapter 3.2.3 --- Sunspot --- p.43 / Chapter 3.3 --- Choices of recurrent neural network parameters and initialization methods --- p.44 / Chapter 3.3.1 --- "Choices of numbers of input, hidden and output units" --- p.45 / Chapter 3.3.2 --- Initial hidden states --- p.45 / Chapter 3.3.3 --- Weight initialization method --- p.45 / Chapter 3.4 --- Method of dealing with over-fitting --- p.47 / Chapter Chapter 4 --- Updating methods --- p.48 / Chapter 4.1 --- Introduction --- p.48 / Chapter 4.2 --- Asynchronous updating method --- p.49 / Chapter 4.2.1 --- Algorithm --- p.49 / Chapter 4.2.2 --- Method of study --- p.50 / Chapter 4.2.3 --- Performance --- p.51 / Chapter 4.2.4 --- Investigation on poor generalization --- p.52 / Chapter 4.2.4.1 --- Hidden states --- p.52 / Chapter 4.2.4.2 --- Incoming weight magnitudes of the hidden units --- p.54 / Chapter 4.2.4.3 --- Weight change against time --- p.56 / Chapter 4.3 --- Asynchronous updating with constraint method --- p.68 / Chapter 4.3.1 --- Algorithm --- p.68 / Chapter 4.3.2 --- Method of study --- p.69 / Chapter 4.3.3 --- Performance --- p.70 / Chapter 4.3.3.1 --- Generalization performance --- p.70 / Chapter 4.3.3.2 --- Training time performance --- p.71 / Chapter 4.3.4 --- Hidden states and incoming weight magnitudes of the hidden units --- p.73 / Chapter 4.3.4.1 --- Hidden states --- p.73 / Chapter 4.3.4.2 --- Incoming weight magnitudes of the hidden units --- p.73 / Chapter 4.4 --- Synchronous updating methods --- p.84 / Chapter 4.4.1 --- Single λ and multiple λ's synchronous updating methods --- p.84 / Chapter 4.4.1.1 --- Algorithm of single λ synchronous updating method --- p.84 / Chapter 4.4.1.2 --- Algorithm of multiple λ's synchronous updating method --- p.85 / Chapter 4.4.1.3 --- Method of study --- p.87 / Chapter 4.4.1.4 --- Performance --- p.87 / Chapter 4.4.1.5 --- Investigation on long training time: analysis of λ --- p.89 / Chapter 4.4.2 --- Multiple λ's with line search synchronous updating method --- p.97 / Chapter 4.4.2.1 --- Algorithm --- p.97 / Chapter 4.4.2.2 --- Performance --- p.98 / Chapter 4.4.2.3 --- Comparison of λ --- p.100 / Chapter 4.5 --- Comparison between asynchronous and synchronous updating methods --- p.101 / Chapter 4.5.1 --- Final training time --- p.101 / Chapter 4.5.2 --- Computation load per complete weight update --- p.102 / Chapter 4.5.3 --- Convergence speed --- p.103 / Chapter 4.6 --- Comparison between our proposed methods and the gradient descent method with adaptive learning rate and momentum --- p.111 / Chapter Chapter 5 --- Number and sizes of the blocks --- p.113 / Chapter 5.1 --- Introduction --- p.113 / Chapter 5.2 --- Performance --- p.113 / Chapter 5.2.1 --- Method of study --- p.113 / Chapter 5.2.2 --- Trend of performance --- p.115 / Chapter 5.2.2.1 --- Asynchronous updating method --- p.115 / Chapter 5.2.2.2 --- Synchronous updating method --- p.116 / Chapter 5.3 --- Computation load per complete weight update --- p.116 / Chapter 5.4 --- Convergence speed --- p.117 / Chapter 5.4.1 --- Trend of inverse of convergence speed --- p.117 / Chapter 5.4.2 --- Factors affecting the convergence speed --- p.117 / Chapter Chapter 6 --- Weight-grouping methods --- p.125 / Chapter 6.1 --- Introduction --- p.125 / Chapter 6.2 --- Training time and generalization performance of different weight-grouping methods --- p.125 / Chapter 6.2.1 --- Method of study --- p.125 / Chapter 6.2.2 --- Performance --- p.126 / Chapter 6.3 --- Degree of approximation of block-diagonal Hessian matrix with different weight- grouping methods --- p.128 / Chapter 6.3.1 --- Method of study --- p.128 / Chapter 6.3.2 --- Performance --- p.128 / Chapter Chapter 7 --- Discussion --- p.150 / Chapter 7.1 --- Advantages and disadvantages of using block-diagonal Hessian matrix --- p.150 / Chapter 7.1.1 --- Advantages --- p.150 / Chapter 7.1.2 --- Disadvantages --- p.151 / Chapter 7.2 --- Analysis of computation complexity --- p.151 / Chapter 7.2.1 --- Trend of computation complexity of each calculation --- p.154 / Chapter 7.2.2 --- Batch mode of updating --- p.155 / Chapter 7.2.3 --- Sequential mode of updating --- p.155 / Chapter 7.3 --- Analysis of storage complexity --- p.156 / Chapter 7.3.1 --- Trend of storage complexity of each set of variables --- p.157 / Chapter 7.3.2 --- Trend of overall storage complexity --- p.157 / Chapter 7.4 --- Parallel implementation --- p.158 / Chapter 7.5 --- Alternative implementation of weight change constraint --- p.158 / Chapter Chapter 8 --- Conclusions --- p.160 / References --- p.162

by Yuen Wai-ying. / Thesis (M.Phil.)--Chinese University of Hong Kong, 1998. / Includes bibliographical references (leaves 50-51). / Abstract also in Chinese. / Chapter Chapter 1 --- Introduction --- p.1 / Chapter Chapter 2 --- Estimation of Factor Scores in a Three-level Factor Analysis Model / Chapter 2.1 --- The Three-level Factor Analysis Model --- p.5 / Chapter 2.2 --- Estimation of Factor Scores in Between-group --- p.7 / Chapter 2.2.1 --- REG Method --- p.9 / Chapter 2.2.2 --- GLS Method --- p.11 / Chapter 2.3 --- Estimation of Factor Scores in Second Level Within-group --- p.13 / Chapter 2.3.1 --- REG Method --- p.15 / Chapter 2.3.2 --- GLS Method --- p.17 / Chapter 2.4 --- Estimation of Factor Scores in First Level Within-group / Chapter 2.4.1 --- First Approach --- p.19 / Chapter 2.4.2 --- Second Approach --- p.24 / Chapter 2.4.3 --- Comparison of the Two Approaches in Estimating Factor Scores in First Level Within-group --- p.31 / Chapter 2.5 --- Summary on the REG and GLS Methods --- p.35 / Chapter Chapter 3 --- Simulation Studies / Example1 --- p.37 / Example2 --- p.42 / Chapter Chapter 4 --- Conclusion and Discussion --- p.48 / References --- p.50 / Figures --- p.52

"Ultrasound has become a widely used form of medical imaging because it is low-cost, safe, and portable. However, it is heavily dependent on the skill of the operator to capture quality images and properly detect abnormalities. Training is a key component of ultrasound, but the limited availability of training courses and programs presents a significant obstacle to the wider use of ultrasound systems. The goal of this work was to design and implement an interactive training system to help train and evaluate sonographers. This Virtual Training System for Diagnostic Ultrasound is an inexpensive, software-based training system in which the trainee scans a generic scan surface with a sham transducer containing position and orientation sensors. The observed ultrasound image is generated from a pre-stored 3D image volume and is controlled interactively by the user€™s movements of the sham transducer. The patient in the virtual environment represented by the 3D image data may depict normal anatomy, exhibit a specific trauma, or present a given physical condition. The training system provides a realistic scanning experience by providing an interactive real-time display with adjustable image parameters similar to those of an actual diagnostic ultrasound system. This system has been designed to limit the amount of hardware needed to allow for low-cost and portability for the user. The system is able to utilize a PC to run the software. To represent the patient to be scanned, a specific scan surface has been produced that allows for an optical sensor to track the position of the sham transducer. The orientation of the sham transducer is tracked by using an inexpensive inertial measurement unit that relies on the use of quaternions to be integrated into the system. The lack of a physical manikin is overcome by using a visual implementation of a virtual patient in the software along with a virtual transducer that reflects the movements of the user on the scan surface. Pre-processing is performed on the selected 3D image volume to provide coordinate transformation parameters that yield a least-mean square fit from the scan surface to the scanning region of the virtual patient. This thesis presents a prototype training system accomplishing the main goals of being low-cost, portable, and accurate. The ultrasound training system can provide cost-effective and convenient training of physicians and sonographers. This system has the potential to become a powerful tool for training sonographers in recognizing a wide variety of medical conditions."