{"id":{"repo_id":"usf","oai_identifier":"oai:digitalcommons.usf.edu:etd-11548"},"canonical_url":"https://search.ndltd.org/etd/usf/oai:digitalcommons.usf.edu:etd-11548","repository":{"repo_id":"usf","name":"University of South Florida","base_url":"https://digitalcommons.usf.edu/do/oai/"},"display":{"title":"Exploring the Use of Enhanced SWAD Towards Building Learned Models that Generalize Better to Unseen Sources","abstract":"Deep learning models, typically, take significant time to train. Classifier ensembles are areliable way to increase classifier accuracy and perhaps generalizability to unseen sources of data. These classifiers can be combined with a simple voting scheme. The problem is that having multiple models can very heavily increase training time. Snapshot ensembles have been shown to provide a boost in performance by creating an ensemble of classifiers with different weights during the training of a single deep learned model. This can somewhat solve the problem of the increased training time as you do not have to train separate models. As Machine Learning becomes a more common tool in the medical field, the domain general- ization problem becomes more important. Domain generalization is a deep-learned model’s ability to generalize to unseen sources. That is, to perform well on sources that were not seen during training. In image recognition and classification, especially in a medical setting, this is important because domain shifts can cause accuracy of medical image classification to drop significantly. Recent work in 2021 proposed a method named Stochastic Weight Av- eraging Densely (SWAD) that, through finding a flatter minima on the loss function during training, decreases the generalization gap between seen and unseen data. In this thesis, we show that the SWAD learning algorithm can increase the accuracy on out-of-domain chest X-ray data by 4.23% (P-value < 0.05) on average. In addition to this finding, an alteration to the SWAD algorithm is introduced that allows an increase in accuracy over the original algorithm on the chest X-ray dataset by 2.27% on average (2 tests P-value < 0.05, 2 tests P-value > 0.05).","abstract_html":"Deep learning models, typically, take significant time to train. Classifier ensembles are areliable way to increase classifier accuracy and perhaps generalizability to unseen sources of data. These classifiers can be combined with a simple voting scheme. The problem is that having multiple models can very heavily increase training time. Snapshot ensembles have been shown to provide a boost in performance by creating an ensemble of classifiers with different weights during the training of a single deep learned model. This can somewhat solve the problem of the increased training time as you do not have to train separate models. As Machine Learning becomes a more common tool in the medical field, the domain general- ization problem becomes more important. Domain generalization is a deep-learned model’s ability to generalize to unseen sources. That is, to perform well on sources that were not seen during training. In image recognition and classification, especially in a medical setting, this is important because domain shifts can cause accuracy of medical image classification to drop significantly. Recent work in 2021 proposed a method named Stochastic Weight Av- eraging Densely (SWAD) that, through finding a flatter minima on the loss function during training, decreases the generalization gap between seen and unseen data. In this thesis, we show that the SWAD learning algorithm can increase the accuracy on out-of-domain chest X-ray data by 4.23% (P-value &lt; 0.05) on average. In addition to this finding, an alteration to the SWAD algorithm is introduced that allows an increase in accuracy over the original algorithm on the chest X-ray dataset by 2.27% on average (2 tests P-value &lt; 0.05, 2 tests P-value &gt; 0.05).","abstract_has_math":false,"creators":["Weinhofer, Brandon M."],"institution":"University of South Florida","degree_name":"MS in Computer Engineering (M.S.C.P.)","degree_level":"Thesis","degree_discipline":"Computer Science and Engineering","degree_department":null,"school":null,"contributors":["Lawrence O. Hall, Ph.D.","Dmitry Goldgof, Ph.D.","Shaun Canavan, Ph.D."],"advisors":[],"committee_chairs":[],"committee_members":[],"year":2024,"date_issued":"2024-03-22T07:00:00Z","date_published":"2024-03-22T07:00:00Z","updated_at":"2026-11-19T08:48:51Z","subjects":["Deep Learning","Domain Generalization","Machine Learning","Snapshot Ensembles","SWAD","Computer Engineering"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.usf.edu/etd/10261","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.usf.edu/etd/10261","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.usf.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.usf.edu%3Aetd-11548","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Lawrence O. Hall, Ph.D.","Dmitry Goldgof, Ph.D.","Shaun Canavan, Ph.D."]},{"key":"dc:creator","label":"Author","values":["Weinhofer, Brandon M."]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date","label":"Dc Date","values":["2024-03-22T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science and Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis","M.S.Cp."]},{"key":"thesis:degree_name","label":"Degree Name","values":["MS in Computer Engineering (M.S.C.P.)"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["University of South Florida"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Deep Learning","Domain Generalization","Machine Learning","Snapshot Ensembles","SWAD","Computer Engineering"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.usf.edu/etd/10261"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description","label":"Description","values":["Deep learning models, typically, take significant time to train. Classifier ensembles are areliable way to increase classifier accuracy and perhaps generalizability to unseen sources of data. These classifiers can be combined with a simple voting scheme. The problem is that having multiple models can very heavily increase training time. Snapshot ensembles have been shown to provide a boost in performance by creating an ensemble of classifiers with different weights during the training of a single deep learned model. This can somewhat solve the problem of the increased training time as you do not have to train separate models. As Machine Learning becomes a more common tool in the medical field, the domain general- ization problem becomes more important. Domain generalization is a deep-learned model’s ability to generalize to unseen sources. That is, to perform well on sources that were not seen during training. In image recognition and classification, especially in a medical setting, this is important because domain shifts can cause accuracy of medical image classification to drop significantly. Recent work in 2021 proposed a method named Stochastic Weight Av- eraging Densely (SWAD) that, through finding a flatter minima on the loss function during training, decreases the generalization gap between seen and unseen data. In this thesis, we show that the SWAD learning algorithm can increase the accuracy on out-of-domain chest X-ray data by 4.23% (P-value < 0.05) on average. In addition to this finding, an alteration to the SWAD algorithm is introduced that allows an increase in accuracy over the original algorithm on the chest X-ray dataset by 2.27% on average (2 tests P-value < 0.05, 2 tests P-value > 0.05)."]},{"key":"dc:title","label":"Title","values":["Exploring the Use of Enhanced SWAD Towards Building Learned Models that Generalize Better to Unseen Sources"]}]}],"canonical_facts":{"dc:contributor":["Lawrence O. Hall, Ph.D.","Dmitry Goldgof, Ph.D.","Shaun Canavan, Ph.D."],"dc:creator":["Weinhofer, Brandon M."],"dc:date":["2024-03-22T07:00:00Z"],"dc:description":["Deep learning models, typically, take significant time to train. Classifier ensembles are areliable way to increase classifier accuracy and perhaps generalizability to unseen sources of data. These classifiers can be combined with a simple voting scheme. The problem is that having multiple models can very heavily increase training time. Snapshot ensembles have been shown to provide a boost in performance by creating an ensemble of classifiers with different weights during the training of a single deep learned model. This can somewhat solve the problem of the increased training time as you do not have to train separate models. As Machine Learning becomes a more common tool in the medical field, the domain general- ization problem becomes more important. Domain generalization is a deep-learned model’s ability to generalize to unseen sources. That is, to perform well on sources that were not seen during training. In image recognition and classification, especially in a medical setting, this is important because domain shifts can cause accuracy of medical image classification to drop significantly. Recent work in 2021 proposed a method named Stochastic Weight Av- eraging Densely (SWAD) that, through finding a flatter minima on the loss function during training, decreases the generalization gap between seen and unseen data. In this thesis, we show that the SWAD learning algorithm can increase the accuracy on out-of-domain chest X-ray data by 4.23% (P-value < 0.05) on average. In addition to this finding, an alteration to the SWAD algorithm is introduced that allows an increase in accuracy over the original algorithm on the chest X-ray dataset by 2.27% on average (2 tests P-value < 0.05, 2 tests P-value > 0.05)."],"dc:identifier":["https://digitalcommons.usf.edu/etd/10261"],"dc:subject":["Deep Learning","Domain Generalization","Machine Learning","Snapshot Ensembles","SWAD","Computer Engineering"],"dc:title":["Exploring the Use of Enhanced SWAD Towards Building Learned Models that Generalize Better to Unseen Sources"],"thesis:degree_discipline":["Computer Science and Engineering"],"thesis:degree_level":["Thesis","M.S.Cp."],"thesis:degree_name":["MS in Computer Engineering (M.S.C.P.)"],"thesis:institution_name":["University of South Florida"]},"updated_at":"2026-11-19T08:48:51Z"}