Back to search

Cal Poly

Empirical Comparisons of Partial Dimension Reduction Algorithms in High-Dimensional Regression

Abstract

dc:description.abstract

In high-dimensional regression problems, dimension reduction methods are often used to address the challenges of multicollinearity and estimation instability. Partial dimension reduction extends these ideas by applying dimension reduction to a subset of the predictors, while the remaining predictors are modeled without compression. This approach is particularly useful when it is important to retain variability and interpretability in certain predictors. This thesis investigates the empirical performance of partial dimension reduction algorithms and introduces a novel algorithm, Iterative Partial Residual (IPR). Two algorithms are considered: a baseline algorithm, Marginal Residual (MR), and the proposed IPR method. Their predictive performance is evaluated using the mean squared error (MSE) of the coefficient estimates across a range of simulation settings, including varying predictor dimensions and covariance structures. Specifically, simulations are conducted under envelope and low-rank spiked covariance structures. The results show that IPR and MR generally exhibit similar predictive performance across most settings. Partial dimension reduction shows the greatest benefit over full dimension reduction under the low-rank spiked covariance structures, while the improvements over full dimension reduction methods are more limited under the envelope covariance structures. However, in some settings with dependence between predictor subsets, the IPR algorithm showed improved estimation of the coefficients associated with the predictors that did not go through dimension reduction. Additionally, convergence issues are observed for the IPR algorithm in certain high-dimensional settings, highlighting practical limitations for application. These results provide an understanding of the strengths and limitations of partial dimension reduction methods in high-dimensional regression.

Degree

thesis:*
Name thesis:degree_name
MS in Statistics
Discipline thesis:degree_discipline
Statistics

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Greenfield, Nathan
Contributors dc:contributor
  • Trevor D. Ruiz
  • Statistics
  • College of Science and Mathematics

Subjects

dc:subject × 3

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.calpoly.edu/theses/3230

Chain of custody

source
Harvested from
Cal Poly
Base URL
digitalcommons.calpoly.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Greenfield, Nathan. Empirical Comparisons of Partial Dimension Reduction Algorithms in High-Dimensional Regression. https://digitalcommons.calpoly.edu/theses/3230