Cal Poly
Empirical Comparisons of Partial Dimension Reduction Algorithms in High-Dimensional Regression
Abstract
dc:description.abstractIn high-dimensional regression problems, dimension reduction methods are often used to address the challenges of multicollinearity and estimation instability. Partial dimension reduction extends these ideas by applying dimension reduction to a subset of the predictors, while the remaining predictors are modeled without compression. This approach is particularly useful when it is important to retain variability and interpretability in certain predictors. This thesis investigates the empirical performance of partial dimension reduction algorithms and introduces a novel algorithm, Iterative Partial Residual (IPR). Two algorithms are considered: a baseline algorithm, Marginal Residual (MR), and the proposed IPR method. Their predictive performance is evaluated using the mean squared error (MSE) of the coefficient estimates across a range of simulation settings, including varying predictor dimensions and covariance structures. Specifically, simulations are conducted under envelope and low-rank spiked covariance structures. The results show that IPR and MR generally exhibit similar predictive performance across most settings. Partial dimension reduction shows the greatest benefit over full dimension reduction under the low-rank spiked covariance structures, while the improvements over full dimension reduction methods are more limited under the envelope covariance structures. However, in some settings with dependence between predictor subsets, the IPR algorithm showed improved estimation of the coefficients associated with the predictors that did not go through dimension reduction. Additionally, convergence issues are observed for the IPR algorithm in certain high-dimensional settings, highlighting practical limitations for application. These results provide an understanding of the strengths and limitations of partial dimension reduction methods in high-dimensional regression.
Degree
thesis:*- Name thesis:degree_name
- MS in Statistics
- Discipline thesis:degree_discipline
- Statistics
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Greenfield, Nathan
- Contributors dc:contributor
-
- Trevor D. Ruiz
- Statistics
- College of Science and Mathematics
Subjects
dc:subject × 3Identifiers
dc:identifier.*- Repository record dc:identifier
- https://digitalcommons.calpoly.edu/theses/3230