{"id":{"repo_id":"mtu","oai_identifier":"oai:digitalcommons.mtu.edu:etdr-2880"},"canonical_url":"https://search.ndltd.org/etd/mtu/oai:digitalcommons.mtu.edu:etdr-2880","repository":{"repo_id":"mtu","name":"Michigan Technological University","base_url":"https://digitalcommons.mtu.edu/do/oai/"},"display":{"title":"A PHYSICAL TEST ARTIFACT FOR EVALUATING EDGE CASES OF INDIVIDUAL AND FUSED AUTOMATED DRIVING PERCEPTION SENSORS","abstract":"With the advent of technologies to support autonomous vehicles (AVs), the number of different AV models from a variety of companies and organizations has proliferated. With this increase in options comes the need to physically evaluate their perception systems. However, there is a lack of standard methods to physically evaluate these perception systems. A set of test artifacts can be used to compare the performances of perception systems, but the artifacts must be usable with different types of perception sensors and various sensor fusion systems. This thesis presents the development of an artifact that injects edge case scenarios into the environment through undetectable and detectable capabilities for light detection and ranging (LiDAR) and radar sensors. Once these artifact capabilities were validated through testing, an evaluation method was developed using this test artifact to compare colored point cloud datasets produced by two LiDAR-camera fusion systems and a stereo camera system. This proposed evaluation method ranks the sensor fusion systems through three metrics that describe the accuracy their colored point cloud representation of both sides of the test artifact. The metrics measure the point cloud's ability to accurately fill the correct amount of space the artifact encompasses in the environment, the spread of the points within this coverage, and the variation in their color values. With this artifact, the evaluation method produced results that followed prior observations, proving the artifact's use of physically comparing different sensor fusion systems.","abstract_html":"With the advent of technologies to support autonomous vehicles (AVs), the number of different AV models from a variety of companies and organizations has proliferated. With this increase in options comes the need to physically evaluate their perception systems. However, there is a lack of standard methods to physically evaluate these perception systems. A set of test artifacts can be used to compare the performances of perception systems, but the artifacts must be usable with different types of perception sensors and various sensor fusion systems. This thesis presents the development of an artifact that injects edge case scenarios into the environment through undetectable and detectable capabilities for light detection and ranging (LiDAR) and radar sensors. Once these artifact capabilities were validated through testing, an evaluation method was developed using this test artifact to compare colored point cloud datasets produced by two LiDAR-camera fusion systems and a stereo camera system. This proposed evaluation method ranks the sensor fusion systems through three metrics that describe the accuracy their colored point cloud representation of both sides of the test artifact. The metrics measure the point cloud&#x27;s ability to accurately fill the correct amount of space the artifact encompasses in the environment, the spread of the points within this coverage, and the variation in their color values. With this artifact, the evaluation method produced results that followed prior observations, proving the artifact&#x27;s use of physically comparing different sensor fusion systems.","abstract_has_math":false,"creators":["Schaefer, Colin"],"institution":null,"degree_name":"Master of Science in Mechanical Engineering (MS)","degree_level":"Open Access Master's Thesis","degree_discipline":"Department of Mechanical Engineering-Engineering Mechanics","degree_department":null,"school":null,"contributors":["Vinh Nguyen"],"advisors":[],"committee_chairs":[],"committee_members":["Jungyun Bae","Nathir Rawashdeh"],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:19:03Z","subjects":["Sensor Evaluation","Metrology","LiDAR","Radar","Camera","Automated Driving","Mechanical Engineering","Other Mechanical Engineering"],"languages":[],"rights":[],"rights_urls":["http://creativecommons.org/licenses/by-nc-nd/4.0/"],"identifier_entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.mtu.edu/etdr/1721"],"render_values":[{"text":"https://digitalcommons.mtu.edu/etdr/1721","href":"https://digitalcommons.mtu.edu/etdr/1721","code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://doi.org/10.37099/mtu.dc.etdr/1721","outbound_label":"DOI","outbound_source":"dc:identifier","record_url":"https://doi.org/10.37099/mtu.dc.etdr/1721","record_label":"Repository record","record_kind":"doi","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.mtu.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.mtu.edu%3Aetdr-2880","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Vinh Nguyen"]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Jungyun Bae","Nathir Rawashdeh"]},{"key":"dc:creator","label":"Author","values":["Schaefer, Colin"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.created","label":"Dc Date Created","values":["2024-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Department of Mechanical Engineering-Engineering Mechanics"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Open Access Master's Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science in Mechanical Engineering (MS)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Sensor Evaluation","Metrology","LiDAR","Radar","Camera","Automated Driving","Mechanical Engineering","Other Mechanical Engineering"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights.license","label":"Dc Rights License","values":["http://creativecommons.org/licenses/by-nc-nd/4.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["10.37099/mtu.dc.etdr/1721","https://digitalcommons.mtu.edu/etdr/1721"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["With the advent of technologies to support autonomous vehicles (AVs), the number of different AV models from a variety of companies and organizations has proliferated. With this increase in options comes the need to physically evaluate their perception systems. However, there is a lack of standard methods to physically evaluate these perception systems. A set of test artifacts can be used to compare the performances of perception systems, but the artifacts must be usable with different types of perception sensors and various sensor fusion systems. This thesis presents the development of an artifact that injects edge case scenarios into the environment through undetectable and detectable capabilities for light detection and ranging (LiDAR) and radar sensors. Once these artifact capabilities were validated through testing, an evaluation method was developed using this test artifact to compare colored point cloud datasets produced by two LiDAR-camera fusion systems and a stereo camera system. This proposed evaluation method ranks the sensor fusion systems through three metrics that describe the accuracy their colored point cloud representation of both sides of the test artifact. The metrics measure the point cloud's ability to accurately fill the correct amount of space the artifact encompasses in the environment, the spread of the points within this coverage, and the variation in their color values. With this artifact, the evaluation method produced results that followed prior observations, proving the artifact's use of physically comparing different sensor fusion systems."]},{"key":"dc:title","label":"Title","values":["A PHYSICAL TEST ARTIFACT FOR EVALUATING EDGE CASES OF INDIVIDUAL AND FUSED AUTOMATED DRIVING PERCEPTION SENSORS"]}]}],"canonical_facts":{"dc:contributor":["Vinh Nguyen"],"dc:contributor.committeemember":["Jungyun Bae","Nathir Rawashdeh"],"dc:creator":["Schaefer, Colin"],"dc:date.created":["2024-01-01T08:00:00Z"],"dc:description.abstract":["With the advent of technologies to support autonomous vehicles (AVs), the number of different AV models from a variety of companies and organizations has proliferated. With this increase in options comes the need to physically evaluate their perception systems. However, there is a lack of standard methods to physically evaluate these perception systems. A set of test artifacts can be used to compare the performances of perception systems, but the artifacts must be usable with different types of perception sensors and various sensor fusion systems. This thesis presents the development of an artifact that injects edge case scenarios into the environment through undetectable and detectable capabilities for light detection and ranging (LiDAR) and radar sensors. Once these artifact capabilities were validated through testing, an evaluation method was developed using this test artifact to compare colored point cloud datasets produced by two LiDAR-camera fusion systems and a stereo camera system. This proposed evaluation method ranks the sensor fusion systems through three metrics that describe the accuracy their colored point cloud representation of both sides of the test artifact. The metrics measure the point cloud's ability to accurately fill the correct amount of space the artifact encompasses in the environment, the spread of the points within this coverage, and the variation in their color values. With this artifact, the evaluation method produced results that followed prior observations, proving the artifact's use of physically comparing different sensor fusion systems."],"dc:identifier":["10.37099/mtu.dc.etdr/1721","https://digitalcommons.mtu.edu/etdr/1721"],"dc:rights.license":["http://creativecommons.org/licenses/by-nc-nd/4.0/"],"dc:subject":["Sensor Evaluation","Metrology","LiDAR","Radar","Camera","Automated Driving","Mechanical Engineering","Other Mechanical Engineering"],"dc:title":["A PHYSICAL TEST ARTIFACT FOR EVALUATING EDGE CASES OF INDIVIDUAL AND FUSED AUTOMATED DRIVING PERCEPTION SENSORS"],"thesis:degree_discipline":["Department of Mechanical Engineering-Engineering Mechanics"],"thesis:degree_level":["Open Access Master's Thesis"],"thesis:degree_name":["Master of Science in Mechanical Engineering (MS)"]},"updated_at":"2026-11-19T08:19:03Z"}