In regression with near collinear explanatory variables, the least squares predictor has large variance. Ordinary least squares regression (OLSR) often leads to unrealistic regression coefficients. Several regularized regression methods have been proposed as alternatives. Well-known are principal components regression (PCR), ridge regression (RR) and continuum regression (CR). The latter two involve a continuous metaparameter, offering additional flexibility. For a univariate response variable, CR incorporates OLSR, PLSR, and PCR as special cases, for special values of the metaparameter. CR is also closely related to RR. However, CR can in fact yield regressors that vary discontinuously with the metaparameter. Thus, the relation between CR and RR is not always one-to-one. We develop a new class of regression methods, LSRR, essentially the same as CR, but without discontinuities, and prove that any optimization principle will yield a regressor proportional to a RR, provided only that the principle implies maximizing some function of the regressor's sample correlation coefficient and its sample variance. For a multivariate response vector we demonstrate that a number of well-established regression methods are related, in that they are special cases of basically one general procedure. We try a more general method based on this procedure, with two meta-parameters. In a simulation study we compare this method to ridge regression, multivariate PLSR and repeated univariate PLSR. For most types of data studied, all methods do approximately equally well. There are cases where RR and LSRR yield larger errors than the other methods, and we conclude that one-factor methods are not adequate for situations where more than one latent variable are needed to describe the data. Among those based on latent variables, none of the methods tried is superior to the others in any obvious way.

In this thesis advanced regression methods are applied to discuss and investigate highly relevant research questions in the areas of finance and economics. In the field of credit risk the thesis investigates a hierarchical model which allows to obtain a consensus score, if several ratings are available for each firm. Autoregressive processes and random effects are used to model both a correlation structure between and within the obligors in the sample. The model also allows to validate the raters themselves. The problem of model uncertainty and multicollinearity between the explanatory variables is addressed in the other two applications. Penalized regressions, like bridge regressions, are used to handle multicollinearity while model averaging techniques allow to account for model uncertainty. The second part of the thesis makes use of Bayesian elastic nets and Bayesian Model Averaging (BMA) techniques to discuss long-term economic growth. It identifies variables which are significantly related to long-term growth. Additionally, it illustrates the superiority of this approach in terms of predictive accuracy. Finally, the third part combines ridge regressions with BMA to identify macroeconomic variables which are significantly related to aggregated firm failure rates. The estimated results deliver important insights for e.g., stress-test scenarios. (author's abstract)

Behavioral data are frequently plagued with highly intercorrelated variables. Collinearity is an indication of insufficient information in the model or in the data. It, therefore, contributes to the unreliability of the estimated coefficients. One result of collinearity is that regression weights derived in one sample may lead to poor prediction in another model. One technique which was developed to deal with highly intercorrelated independent variables is ridge regression. It was first proposed by Hoerl and Kennard in 1970 as a method which would allow the data analyst to both stabilize his estimates and improve upon his squared error loss. The problem of this study was the application of ridge regression in the analysis of data resulting from educational research.

Multiple linear regression model plays a key role in statistical inference and it has extensive applications in business, environmental, physical and social sciences. Multicollinearity has been a considerable problem in multiple regression analysis. When the regressor variables are multicollinear, it becomes difficult to make precise statistical inferences about the regression coefficients. There are some statistical methods that can be used, which are discussed in this thesis are ridge regression, Liu, two parameter biased and LASSO estimators. Firstly, an analytical comparison on the basis of risk was made among ridge, Liu and LASSO estimators under orthonormal regression model. I found that LASSO dominates least squares, ridge and Liu estimators over a significant portion of the parameter space for large dimension. Secondly, a simulation study was conducted to compare performance of ridge, Liu and two parameter biased estimator by their mean squared error criterion. I found that two parameter biased estimator performs better than its corresponding ridge regression estimator. Overall, Liu estimator performs better than both ridge and two parameter biased estimator.

In the development of materials, the understanding of their properties is crucial. For magnetic materials, magnetism is an apparent property that needs to be accounted for. There are multiple factors explaining the phenomenon of magnetism, one being the effect of vibrations of the atoms on longitudinal spin fluctuations. This effect can be investigated by simulations, using density functional theory, and calculating energy landscapes. Through such simulations, the energy landscapes have been found to depend on the magnetic background and the positions of the atoms. However, when simulating a supercell of many atoms, to calculate energy landscapes for all atoms consumes many hours on the supercomputer. In this thesis, the possibility of using machine learning models to accelerate the approximation of energy landscapes is investigated. The material under investigation is body-centered cubic iron in the paramagnetic state at 1043 K. Machine learning enables statistical predictions to be made on new data based on patterns found in a previous set of data. Kernel ridge regression is used as the machine learning method. An important issue when training a machine learning model is the representation of the data in the so called descriptor (feature vector representation) or, more specific to this case, how the environment of an atom in a supercell is accounted for and represented properly. Four different descriptors are developed and compared to investigate which one yields the best result and why. Apart from comparing the descriptors, the results when using machine learning models are compared to when using other methods to approximate the energy landscapes. The machine learning models are also tested in a combined atomistic spin dynamics and ab initio molecular dynamics simulation (ASD-AIMD) where they were used to approximate energy landscapes and, from that, magnetic moment magnitudes at 1043 K. The results of these simulations are compared to the results from two other cases: one where the magnetic moment magnitudes are set to a constant value and one where they are set to their magnitudes at 0 K. From these investigations it is found that using machine learning methods to approximate the energy landscapes does, to a large degree, decrease the errors compared to the other approximation methods investigated. Some weaknesses of the respective descriptors were detected and if, in future work, these are accounted for, the errors have the potential of being lowered further.

Motion is one of the most important features describing an image sequence. Motion estimation has been widely applied in structure from motion, vision-based navigation and many other fields. However, real-time motion estimation remains a challenge because of its high computational expense. The traditional CPU-based scheme cannot satisfy the power, size and computation requirements in many applications. With the availability of new parallel architectures such as FPGAs and GPUs, applying these new technologies to computer vision tasks such as motion estimation has been an active research field in recent years. In this dissertation, FPGAs have been applied to real-time motion estimation for their outstanding properties in computation power, size, power consumption and reconfigurability. It is believed in this dissertation that simply migrating the software-based algorithms and mapping them to a specific architecture is not enough to achieve good performance. Accuracy is usually compromised as the cost of migration. Improvement and optimization at the algorithm level are critical to performance. To improve motion estimation on the FPGA platform and prove the effectiveness of the method, three main efforts have been made in the dissertation. First, a lightweight tensor-based algorithm has been designed which can be implemented in a fully pipelined structure. Key factors determining the algorithm performance are analyzed from the simulation results. Second, an improved algorithm is then developed based on the analyses of the first algorithm. This algorithm applies a ridge estimator and temporal smoothing in order to improve the accuracy. A structure composed of two pipelines is designed to accommodate the new algorithm while using reasonable hardware resources. Third, a hardware friendly algorithm is developed to analyze the optical flow field and detect obstacles for unmanned ground vehicle applications. The motion component is de-rotated, de-translated and postprocessed to detect obstacles. All these steps can be efficiently implemented in FPGAs. The characteristics of the FPGA architecture are taken into account in all development processes of these three algorithms. This dissertation also discusses some important perspectives for FPGA-based design in different chapters. These perspectives include software simulation and optimization at the algorithm development stage, hardware simulation and test bench design at the hardware development stage. They are important and particular for the development of FPGA-based computer vision algorithms. The experimental results have shown that the proposed motion estimation module can perform in real-time and achieve over 50% improvement in the motion estimation accuracy compared to the previous work in the literature. The results also show that the motion field can be reliably applied to obstacle detection tasks.

No description available.

Due to recent advances in technology that facilitate acquisition of multi-parameter defined phenotypes, new opportunities have arisen for predicting patient outcomes based on individual specific cell subset changes. The data resulting from these trials can be a challenge to analyze, as predictors may be highly correlated with each other or related to outcome within levels of other predictor variables. As a result, applying traditional methods like simple linear models and univariate approaches such as odds ratios may be insufficient. In this dissertation, we describe potential solutions including tree-based methods, ridge regression, mixed modeling, and a new estimator called a mixed ridge estimator with expectation-maximization (EM) algorithm. Data examples are provided. In particular, flow cytometry is a method of measuring a large number of particle counts at once by suspending them in a fluid and shining a beam of light onto the fluid. This is specifically relevant in the context of studying human immunodeficiency virus (HIV), where there exists a great potential to draw from the rich array of data on host cell-mediated response to infection and drug exposures, to inform and discover patient level determinants of disease progression and/or response to anti-retroviral therapy (ART). The data sets collected are often high dimensional with correlated columns, which can be challenging to analyze. We demonstrate the application and comparative interpretations of three tree-based algorithms for the analysis of data arising from flow cytometry in the first chapter of this manuscript. Specifically, we consider the question of what best predicts CD4 T-cell recovery in HIV-1 infected persons starting antiretroviral therapy with CD4 count between 200-350 cell/μl. The tree-based approaches, namely, classification and regression trees (CART), random forests (RF) and logic regression (LR), were designed specifically to uncover complex structure in high dimensional data settings. While contingency table analysis and RFs provide information on the importance of each potential predictor variable, CART and LR offer additional insight into the combinations of variables that together are predictive of the outcome. Specifically, application of tree-based methods to our data suggest that a combination of baseline immune activation states, with emphasis on CD8 T cell activation, may be a better predictor than any single T cell/innate cell subset analyzed. In the following chapter, tree-based methods are compared to each other via a simulation study. Each has its merits in particular circumstances; for example, RF is able to identify the order of importance of predictors regardless of whether there is a tree-like structure. It is able to adjust for correlation among predictors by using a machine learning algorithm, analyzing subsets of predictors and subjects over a number of iterations. CART is useful when variables are predictive of outcome within levels of other variables, and is able to find the most parsimonious model using pruning. LR also identifies structure within the set of predictor variables, and nicely illustrates relationship among variables. However, due to the vast number of combinations of predictor variables that would need to be analyzed in order to find the single best LR tree, an algorithm is used that only searches a subset of potential combinations of predictors. Therefore, results may be different each time the algorithm is used on the same data set. Next we use a regression approach to analyzing data with correlated predictors. Ridge regression is a method of accounting for correlated data by adding a shrinkage component to the estimators for a linear model. We perform a simulation study to compare ridge regression to linear regression over various correlation coefficients and find that ridge regression outperforms linear regression as correlation increases. To account for collinearity among the predictors along with longitudinal data, a new estimator that combines the applicability of ridge regression and mixed models using an EM algorithm is developed and compared to the mixed model. We find from a simulation study comparing our mixed ridge (MR) approach with a traditional mixed model that our new mixed ridge estimator is able to handle collinearity of predictor variables better than the mixed model, while accounting for random within-subject effects that regular ridge regression does not take into account. As correlation among predictors increases, power decreases more quickly for the mixed model than MR. Additionally, type I error rate is not significantly elevated when the MR approach is taken. The MR estimator gives us new insight into flow cytometry data and other data sets with correlated predictor variables that our tree-based methods could not give us. These methods all provide unique insight into our data that more traditional methods of analysis do not offer.

Multiple regression models play an important role in analyzing and making predictions about data. Prediction accuracy becomes lower when two or more explanatory variables in the model are highly correlated. One solution is to use ridge regression. The purpose of this thesis is to study the performance of available ridge regression estimators for Poisson regression models in the presence of moderately to highly correlated variables. As performance criteria, we use mean square error (MSE), mean absolute percentage error (MAPE), and percentage of times the maximum likelihood (ML) estimator produces a higher MSE than the ridge regression estimator. A Monte Carlo simulation study was conducted to compare performance of the estimators under three experimental conditions: correlation, sample size, and intercept. It is evident from simulation results that all ridge estimators performed better than the ML estimator. We proposed new estimators based on the results, which performed very well compared to the original estimators. Finally, the estimators are illustrated using data on recreational habits.

Through an analysis of the Swedish state budget, models have been developed to deepen the understanding of the effects that government expenditures have on reducing crime. This has been modeled by examining selected crime categories using the mathematical methods Ridge Regression, Lasso Regression and Principal Component Analysis. Combined with a qualitative study of previous research on the economic aspects of crime, an analysis has been conducted. The mathematical methods indicate that it may be more effective to invest in crime prevention measures, such as increased social protection and focus on vulnerable groups, rather than more direct efforts such as increased resources for the police force. However, the result contradicts some of the accepted economic conclusions on the subject, as these highlight the importance of increasing the number of police officers and harsher penalties. These do however also mention the importance of crime prevention measures such as reducing the gaps in society, which is in line with the results of this work. The conclusion should however be used with caution as the models are based on a number of assumptions and could be improved upon further analysis of these, together with more data points that would strengthen the validity of the analysis more. / Genom en analys av Sveriges statsbudget har modeller tagits fram för att försöka förstå de effekter olika samhällssatsningar har på brottslighet i Sverige. Detta har modellerats genom att undersöka utvalda brottskategorier med hjälp av de matematiska metoderna Ridge Regression, Lasso Regression samt Principal Component Analysis. Tillsammans med en kvalitativ undersökning av tidigare forskning gällande nationalekonomiska aspekter kring brottslighet har en analys sedan genomförts. De matematiska metoderna tyder på att det kan vara mer effektivt att satsa på brottsförebyggande åtgärder, såsom ökat socialt skydd och fokus på utsatta grupper, istället för mer direkta satsningar på brottsförhindrande åtgärder som exempelvis ökade resurser till polisväsendet. Däremot motsäger resultatet en del av de vedertagna nationalekonomiska slutsatserna om ämnet, då dessa belyser vikten av ökade antalet poliser och hårdare straff. De lyfter även fram vikten av brottsförebyggande åtgärder såsom att minska klyftorna i samhället, vilket går i linje med resultatet av detta arbete. Slutsatsen ska dock användas med försiktighet då modellerna bygger på flertalet antaganden och skulle kunna förbättras vid ytterligare analys utav dessa, tillsammans med fler datapunkter som skulle stärka validiteten.