{"id":{"repo_id":"bu","oai_identifier":"oai:open.bu.edu:2144/49253"},"canonical_url":"https://search.ndltd.org/etd/bu/oai:open.bu.edu:2144/49253","repository":{"repo_id":"bu","name":"Boston University","base_url":"https://open.bu.edu/oai/request"},"display":{"title":"Auxiliaries for training deep neural networks","abstract":"The generalization of deep neural networks (DNNs) remains a critical challenge in machine learning, where the goal is to ensure that models perform well on unseen test data. Regularization techniques play a pivotal role in this process by constraining the model's capacity to prevent overfitting. Methods such as dropout, data augmentation, normalization, and penalty functions introduce beneficial biases that guide the optimization process towards better generalization. Traditional regularizations are typically data-agnostic. In this dissertation, we propose leveraging data-dependent auxiliary sources for regularization, focusing on image classification across various settings including supervised learning, knowledge distillation, federated learning, meta-learning, and few-shot learning, as well as imitation learning for autonomous driving. We introduce four main types of auxiliary sources: models, gradients, features and data. Auxiliary models approximate the mean of historical predictions, providing data-dependent regularization by aligning the deployed model's output with the auxiliary model's output. This method, which we extend to knowledge distillation, demonstrates improved generalization and draws connections to applications in large language models (LLMs). Auxiliary gradients address bias in neural networks handling multiple tasks, particularly in personalized federated learning and online meta-learning. We propose algorithms that dynamically modify gradients to reduce task-specific biases, enhancing model performance across diverse users and tasks. Then, auxiliary features focus on tasks like few-shot learning and fine-grained image recognition, where specialized modeling is required to capture detailed visual characteristics. We propose Deep Object Parsing to generate auxiliary features that enhance performance in these settings. For imitation learning in autonomous driving, we introduce frameworks that utilize vast amounts of auxiliary data. The SelfD framework leverages unlabeled dash camera data through semi-supervised learning, while the AnyD framework utilizes cross-domain driving data to improve policy learning. Both frameworks demonstrate significant improvements in driving performance by efficiently using auxiliary data.","abstract_html":"The generalization of deep neural networks (DNNs) remains a critical challenge in machine learning, where the goal is to ensure that models perform well on unseen test data. Regularization techniques play a pivotal role in this process by constraining the model&#x27;s capacity to prevent overfitting. Methods such as dropout, data augmentation, normalization, and penalty functions introduce beneficial biases that guide the optimization process towards better generalization. Traditional regularizations are typically data-agnostic. In this dissertation, we propose leveraging data-dependent auxiliary sources for regularization, focusing on image classification across various settings including supervised learning, knowledge distillation, federated learning, meta-learning, and few-shot learning, as well as imitation learning for autonomous driving. We introduce four main types of auxiliary sources: models, gradients, features and data. Auxiliary models approximate the mean of historical predictions, providing data-dependent regularization by aligning the deployed model&#x27;s output with the auxiliary model&#x27;s output. This method, which we extend to knowledge distillation, demonstrates improved generalization and draws connections to applications in large language models (LLMs). Auxiliary gradients address bias in neural networks handling multiple tasks, particularly in personalized federated learning and online meta-learning. We propose algorithms that dynamically modify gradients to reduce task-specific biases, enhancing model performance across diverse users and tasks. Then, auxiliary features focus on tasks like few-shot learning and fine-grained image recognition, where specialized modeling is required to capture detailed visual characteristics. We propose Deep Object Parsing to generate auxiliary features that enhance performance in these settings. For imitation learning in autonomous driving, we introduce frameworks that utilize vast amounts of auxiliary data. The SelfD framework leverages unlabeled dash camera data through semi-supervised learning, while the AnyD framework utilizes cross-domain driving data to improve policy learning. Both frameworks demonstrate significant improvements in driving performance by efficiently using auxiliary data.","abstract_has_math":false,"creators":["Zhu, Ruizhao"],"institution":"Boston University","degree_name":"Doctor of Philosophy","degree_level":"doctoral","degree_discipline":"Electrical & Computer Engineering","degree_department":null,"school":null,"contributors":[],"advisors":["Saligrama, Venkatesh"],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024","date_published":"2024","updated_at":"2026-11-18T19:52:29Z","subjects":["Electrical engineering"],"languages":["en_US"],"rights":["Attribution 4.0 International"],"rights_urls":["http://creativecommons.org/licenses/by/4.0/"],"identifier_entries":[],"person_identifier_entries":[{"key":"dc:identifier.orcid","label":"ORCID","values":["0009-0001-9496-3144"],"render_values":[{"text":"0009-0001-9496-3144","href":"https://orcid.org/0009-0001-9496-3144","code":true}]}]},"links":{"outbound_url":"https://hdl.handle.net/2144/49253","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"https://hdl.handle.net/2144/49253","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://open.bu.edu/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Aopen.bu.edu%3A2144%2F49253","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.advisor","label":"Advisor","values":["Saligrama, Venkatesh"]},{"key":"dc:creator","label":"Author","values":["Zhu, Ruizhao"]},{"key":"dc:identifier.orcid","label":"ORCID","values":["0009-0001-9496-3144"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2024-09-13T14:32:22Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2024-09-13T14:32:22Z"]},{"key":"dc:date.issued","label":"Date","values":["2024"]},{"key":"dc:date.updated","label":"Dc Date Updated","values":["2024-09-11T01:02:04Z"]},{"key":"dc:type","label":"Dc Type","values":["Thesis/Dissertation"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical & Computer Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["doctoral"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Doctor of Philosophy"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Boston University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Electrical engineering"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]},{"key":"dc:rights","label":"Dc Rights","values":["Attribution 4.0 International"]},{"key":"dc:rights.uri","label":"Rights URI","values":["http://creativecommons.org/licenses/by/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/2144/49253"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["The generalization of deep neural networks (DNNs) remains a critical challenge in machine learning, where the goal is to ensure that models perform well on unseen test data. Regularization techniques play a pivotal role in this process by constraining the model's capacity to prevent overfitting. Methods such as dropout, data augmentation, normalization, and penalty functions introduce beneficial biases that guide the optimization process towards better generalization. Traditional regularizations are typically data-agnostic. In this dissertation, we propose leveraging data-dependent auxiliary sources for regularization, focusing on image classification across various settings including supervised learning, knowledge distillation, federated learning, meta-learning, and few-shot learning, as well as imitation learning for autonomous driving. We introduce four main types of auxiliary sources: models, gradients, features and data. Auxiliary models approximate the mean of historical predictions, providing data-dependent regularization by aligning the deployed model's output with the auxiliary model's output. This method, which we extend to knowledge distillation, demonstrates improved generalization and draws connections to applications in large language models (LLMs). Auxiliary gradients address bias in neural networks handling multiple tasks, particularly in personalized federated learning and online meta-learning. We propose algorithms that dynamically modify gradients to reduce task-specific biases, enhancing model performance across diverse users and tasks. Then, auxiliary features focus on tasks like few-shot learning and fine-grained image recognition, where specialized modeling is required to capture detailed visual characteristics. We propose Deep Object Parsing to generate auxiliary features that enhance performance in these settings. For imitation learning in autonomous driving, we introduce frameworks that utilize vast amounts of auxiliary data. The SelfD framework leverages unlabeled dash camera data through semi-supervised learning, while the AnyD framework utilizes cross-domain driving data to improve policy learning. Both frameworks demonstrate significant improvements in driving performance by efficiently using auxiliary data."]},{"key":"dc:title","label":"Title","values":["Auxiliaries for training deep neural networks"]}]}],"canonical_facts":{"dc:contributor.advisor":["Saligrama, Venkatesh"],"dc:creator":["Zhu, Ruizhao"],"dc:date.accessioned":["2024-09-13T14:32:22Z"],"dc:date.available":["2024-09-13T14:32:22Z"],"dc:date.issued":["2024"],"dc:date.updated":["2024-09-11T01:02:04Z"],"dc:description.abstract":["The generalization of deep neural networks (DNNs) remains a critical challenge in machine learning, where the goal is to ensure that models perform well on unseen test data. Regularization techniques play a pivotal role in this process by constraining the model's capacity to prevent overfitting. Methods such as dropout, data augmentation, normalization, and penalty functions introduce beneficial biases that guide the optimization process towards better generalization. Traditional regularizations are typically data-agnostic. In this dissertation, we propose leveraging data-dependent auxiliary sources for regularization, focusing on image classification across various settings including supervised learning, knowledge distillation, federated learning, meta-learning, and few-shot learning, as well as imitation learning for autonomous driving. We introduce four main types of auxiliary sources: models, gradients, features and data. Auxiliary models approximate the mean of historical predictions, providing data-dependent regularization by aligning the deployed model's output with the auxiliary model's output. This method, which we extend to knowledge distillation, demonstrates improved generalization and draws connections to applications in large language models (LLMs). Auxiliary gradients address bias in neural networks handling multiple tasks, particularly in personalized federated learning and online meta-learning. We propose algorithms that dynamically modify gradients to reduce task-specific biases, enhancing model performance across diverse users and tasks. Then, auxiliary features focus on tasks like few-shot learning and fine-grained image recognition, where specialized modeling is required to capture detailed visual characteristics. We propose Deep Object Parsing to generate auxiliary features that enhance performance in these settings. For imitation learning in autonomous driving, we introduce frameworks that utilize vast amounts of auxiliary data. The SelfD framework leverages unlabeled dash camera data through semi-supervised learning, while the AnyD framework utilizes cross-domain driving data to improve policy learning. Both frameworks demonstrate significant improvements in driving performance by efficiently using auxiliary data."],"dc:identifier.orcid":["0009-0001-9496-3144"],"dc:identifier.uri":["https://hdl.handle.net/2144/49253"],"dc:language.iso":["en_US"],"dc:rights":["Attribution 4.0 International"],"dc:rights.uri":["http://creativecommons.org/licenses/by/4.0/"],"dc:subject":["Electrical engineering"],"dc:title":["Auxiliaries for training deep neural networks"],"dc:type":["Thesis/Dissertation"],"thesis:degree_discipline":["Electrical & Computer Engineering"],"thesis:degree_level":["doctoral"],"thesis:degree_name":["Doctor of Philosophy"],"thesis:institution_name":["Boston University"]},"updated_at":"2026-11-18T19:52:29Z"}