{"id":{"repo_id":"calpoly","oai_identifier":"oai:digitalcommons.calpoly.edu:theses-4888"},"canonical_url":"https://search.ndltd.org/etd/calpoly/oai:digitalcommons.calpoly.edu:theses-4888","repository":{"repo_id":"calpoly","name":"Cal Poly","base_url":"https://digitalcommons.calpoly.edu/do/oai/"},"display":{"title":"Empirical Comparisons of Partial Dimension Reduction Algorithms in High-Dimensional Regression","abstract":"In high-dimensional regression problems, dimension reduction methods are often used to address the challenges of multicollinearity and estimation instability. Partial dimension reduction extends these ideas by applying dimension reduction to a subset of the predictors, while the remaining predictors are modeled without compression. This approach is particularly useful when it is important to retain variability and interpretability in certain predictors. This thesis investigates the empirical performance of partial dimension reduction algorithms and introduces a novel algorithm, Iterative Partial Residual (IPR). Two algorithms are considered: a baseline algorithm, Marginal Residual (MR), and the proposed IPR method. Their predictive performance is evaluated using the mean squared error (MSE) of the coefficient estimates across a range of simulation settings, including varying predictor dimensions and covariance structures. Specifically, simulations are conducted under envelope and low-rank spiked covariance structures. The results show that IPR and MR generally exhibit similar predictive performance across most settings. Partial dimension reduction shows the greatest benefit over full dimension reduction under the low-rank spiked covariance structures, while the improvements over full dimension reduction methods are more limited under the envelope covariance structures. However, in some settings with dependence between predictor subsets, the IPR algorithm showed improved estimation of the coefficients associated with the predictors that did not go through dimension reduction. Additionally, convergence issues are observed for the IPR algorithm in certain high-dimensional settings, highlighting practical limitations for application. These results provide an understanding of the strengths and limitations of partial dimension reduction methods in high-dimensional regression.","abstract_html":"In high-dimensional regression problems, dimension reduction methods are often used to address the challenges of multicollinearity and estimation instability. Partial dimension reduction extends these ideas by applying dimension reduction to a subset of the predictors, while the remaining predictors are modeled without compression. This approach is particularly useful when it is important to retain variability and interpretability in certain predictors. This thesis investigates the empirical performance of partial dimension reduction algorithms and introduces a novel algorithm, Iterative Partial Residual (IPR). Two algorithms are considered: a baseline algorithm, Marginal Residual (MR), and the proposed IPR method. Their predictive performance is evaluated using the mean squared error (MSE) of the coefficient estimates across a range of simulation settings, including varying predictor dimensions and covariance structures. Specifically, simulations are conducted under envelope and low-rank spiked covariance structures. The results show that IPR and MR generally exhibit similar predictive performance across most settings. Partial dimension reduction shows the greatest benefit over full dimension reduction under the low-rank spiked covariance structures, while the improvements over full dimension reduction methods are more limited under the envelope covariance structures. However, in some settings with dependence between predictor subsets, the IPR algorithm showed improved estimation of the coefficients associated with the predictors that did not go through dimension reduction. Additionally, convergence issues are observed for the IPR algorithm in certain high-dimensional settings, highlighting practical limitations for application. These results provide an understanding of the strengths and limitations of partial dimension reduction methods in high-dimensional regression.","abstract_has_math":false,"creators":["Greenfield, Nathan"],"institution":null,"degree_name":"MS in Statistics","degree_level":null,"degree_discipline":"Statistics","degree_department":null,"school":null,"contributors":["Trevor D. Ruiz","Statistics","College of Science and Mathematics"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T07:12:51Z","subjects":["Regression problems","partial dimension reduction","Statistical Methodology"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.calpoly.edu/theses/3230","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.calpoly.edu/theses/3230","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.calpoly.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.calpoly.edu%3Atheses-4888","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Trevor D. Ruiz","Statistics","College of Science and Mathematics"]},{"key":"dc:creator","label":"Author","values":["Greenfield, Nathan"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2026-03-11T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2026-03-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Statistics"]},{"key":"thesis:degree_name","label":"Degree Name","values":["MS in Statistics"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Regression problems","partial dimension reduction","Statistical Methodology"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.calpoly.edu/theses/3230"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["In high-dimensional regression problems, dimension reduction methods are often used to address the challenges of multicollinearity and estimation instability. Partial dimension reduction extends these ideas by applying dimension reduction to a subset of the predictors, while the remaining predictors are modeled without compression. This approach is particularly useful when it is important to retain variability and interpretability in certain predictors. This thesis investigates the empirical performance of partial dimension reduction algorithms and introduces a novel algorithm, Iterative Partial Residual (IPR). Two algorithms are considered: a baseline algorithm, Marginal Residual (MR), and the proposed IPR method. Their predictive performance is evaluated using the mean squared error (MSE) of the coefficient estimates across a range of simulation settings, including varying predictor dimensions and covariance structures. Specifically, simulations are conducted under envelope and low-rank spiked covariance structures. The results show that IPR and MR generally exhibit similar predictive performance across most settings. Partial dimension reduction shows the greatest benefit over full dimension reduction under the low-rank spiked covariance structures, while the improvements over full dimension reduction methods are more limited under the envelope covariance structures. However, in some settings with dependence between predictor subsets, the IPR algorithm showed improved estimation of the coefficients associated with the predictors that did not go through dimension reduction. Additionally, convergence issues are observed for the IPR algorithm in certain high-dimensional settings, highlighting practical limitations for application. These results provide an understanding of the strengths and limitations of partial dimension reduction methods in high-dimensional regression."]},{"key":"dc:title","label":"Title","values":["Empirical Comparisons of Partial Dimension Reduction Algorithms in High-Dimensional Regression"]}]}],"canonical_facts":{"dc:contributor":["Trevor D. Ruiz","Statistics","College of Science and Mathematics"],"dc:creator":["Greenfield, Nathan"],"dc:date.available":["2026-03-11T07:00:00Z"],"dc:date.created":["2026-03-01T08:00:00Z"],"dc:description.abstract":["In high-dimensional regression problems, dimension reduction methods are often used to address the challenges of multicollinearity and estimation instability. Partial dimension reduction extends these ideas by applying dimension reduction to a subset of the predictors, while the remaining predictors are modeled without compression. This approach is particularly useful when it is important to retain variability and interpretability in certain predictors. This thesis investigates the empirical performance of partial dimension reduction algorithms and introduces a novel algorithm, Iterative Partial Residual (IPR). Two algorithms are considered: a baseline algorithm, Marginal Residual (MR), and the proposed IPR method. Their predictive performance is evaluated using the mean squared error (MSE) of the coefficient estimates across a range of simulation settings, including varying predictor dimensions and covariance structures. Specifically, simulations are conducted under envelope and low-rank spiked covariance structures. The results show that IPR and MR generally exhibit similar predictive performance across most settings. Partial dimension reduction shows the greatest benefit over full dimension reduction under the low-rank spiked covariance structures, while the improvements over full dimension reduction methods are more limited under the envelope covariance structures. However, in some settings with dependence between predictor subsets, the IPR algorithm showed improved estimation of the coefficients associated with the predictors that did not go through dimension reduction. Additionally, convergence issues are observed for the IPR algorithm in certain high-dimensional settings, highlighting practical limitations for application. These results provide an understanding of the strengths and limitations of partial dimension reduction methods in high-dimensional regression."],"dc:identifier":["https://digitalcommons.calpoly.edu/theses/3230"],"dc:subject":["Regression problems","partial dimension reduction","Statistical Methodology"],"dc:title":["Empirical Comparisons of Partial Dimension Reduction Algorithms in High-Dimensional Regression"],"thesis:degree_discipline":["Statistics"],"thesis:degree_name":["MS in Statistics"]},"updated_at":"2026-11-19T07:12:51Z"}